JP4701111B2 - Pattern matching system and subject tracking system - Google Patents

Description

  The present invention relates to a pattern matching system that determines the degree of coincidence of a plurality of images, and a subject tracking system that tracks the movement of a specific subject in captured images that are continuously captured using the pattern matching system.

  There is known a pattern matching system that determines the coincidence of a plurality of images. Pattern matching is also used for tracking a moving subject in a plurality of images that are continuously captured.

  In the pattern matching system, images are divided finely, the luminance of divided regions having the same relative position in both images is compared, and the matching of the images is determined by the number of regions having the same luminance.

  The more areas to be compared, the higher the accuracy of determination. On the other hand, the more areas to be compared, the digital matching unit having a larger number of bits to be handled needs to be used for the pattern matching system. On the other hand, even if it was possible to handle more bits than necessary, it could be wasted without being used.

  Therefore, an object of the present invention is to provide a pattern matching system that has high calculation efficiency and can effectively use a digital calculator.

  The pattern matching system of the present invention is a pattern matching system that outputs a determination value for indicating the degree of coincidence between a first image and a second image, and forms first and second images. An acquisition unit that acquires, as digital data, first and second image data corresponding to the first and second images formed by region signals whose signal intensity changes according to the luminance of each of the power-of-two pattern regions; The absolute value of the difference between the signal strengths of the region signals in the comparison by the comparison unit that compares the signal strengths of the region signals of the pattern regions at the same position in the first and second images is equal to or less than a predetermined reference value. A determination unit that calculates a determination value that changes according to the number of pattern regions and an output unit that outputs the determination value are provided.

  Further, a binarization unit for binarizing the signal strength of the region signal in the first and second image data is provided, and the comparison unit 2 of the region signal of the pattern region at the same position in the first and second images. It is preferable to compare the quantified signal strengths.

  In addition, it is preferable to increase the size of the pattern region when the luminance of the entire first image or the second image is lower than the first threshold value.

  The acquisition unit receives a pixel signal whose signal intensity changes according to the luminance of each of the plurality of pixels forming the pattern region, and the region signal based on the pixel signal of the plurality of pixels forming the single pattern region. A large pattern formed by a pattern region whose size is enlarged when the luminance of the first image or the entire second image is lower than the first threshold value. A large area signal whose signal intensity changes according to the luminance of the entire area is generated, and the comparison unit determines whether the first image or the second image has a luminance lower than the first threshold value when the luminance of the first image or the entire second image is lower than the first threshold. The signal intensity of large area signals in the large pattern area at the same position is compared, and the decision unit calculates a decision value that changes according to the number of large pattern areas where the absolute value of the difference in signal intensity of the large area signal is below the reference value Preferably

  The subject tracking system of the present invention is a subject tracking system that tracks the movement of a tracking target object, which is a specific subject, in captured images that are continuously captured, and tracks a predetermined area in the captured image. An initial setting unit that initially sets a tracking area for tracking the object, and a candidate area setting unit that sets first and second candidate areas that are areas moved in the first and second directions from the tracking area; The reference image that is the image of the tracking region in the captured image captured at the first timing is detected, and the first and second candidate regions in the captured image captured at the second timing after the first timing are detected. An image detection unit for detecting first and second candidate images, and a power of 2 for forming the reference image and the first and second candidate images based on the reference image and the first and second candidate images Pattern areas Compare the signal intensity of the area signal of the pattern area at the same position in the reference image and the first image or the second image, with the area signal generation unit that generates an area signal whose signal intensity changes according to each luminance. A comparison unit, a first determination unit that calculates a determination value that changes according to the number of pattern regions in which the absolute value of the difference in signal strength of the region signals is equal to or less than a predetermined reference value in the comparison by the comparison unit, and a reference It is determined whether the moving direction of the tracking target object from the first timing to the second timing is the first or second direction based on the determination value for the image and the first image or the second image. And a resetting unit that resets the candidate area that has moved in the direction determined as the movement direction of the tracking target object as the tracking area.

  In addition, the candidate area setting unit extends from the tracking area to the first and second candidate areas in accordance with the length of the focal length of the photographing optical system for forming an optical image of the tracking target object on the light receiving surface of the image sensor. It is preferable to set a long movement distance.

  In addition, an exposure adjustment system is provided that can adjust the exposure when the optical image of the tracking target is received by the imaging device according to the luminance of a part of the optical image incident on the imaging device, and the luminance of the pattern area is the exposure amount. It is preferably used as the luminance of a part of the optical image for performing the adjustment.

  According to the present invention, since pattern matching is performed based on power-of-two pieces of information, it is possible to efficiently use a digital computing unit used in a pattern matching system.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram schematically showing an internal configuration of a digital camera having a subject tracking system that tracks a subject by a pattern matching system to which an embodiment of the present invention is applied.

  The digital camera 10 includes a photographing optical system 11, an image sensor 12, an AFE (Analog Front End) 13, a DSP (Digital Signal Processor) 14, an operation input unit 15, a zoom drive unit 16, a focus drive unit 17, and the like. .

  The photographing optical system 11 is optically connected to the image sensor 12. An optical image of a subject passing through the photographing optical system 11 is incident on the light receiving surface of the image sensor 12. The image sensor 12 is, for example, a CCD. When the optical image of the subject is received on the light receiving surface, an image signal corresponding to the optical image is generated.

  The photographing optical system 11 includes a plurality of lenses such as a variable power lens 11a and a focus lens 11b. The variable magnification lens 11a and the focus lens 11b are movable in the optical axis direction.

  A zoom optical system is formed by the variable magnification lens 11a and the focus lens 11b, and the focal length of the photographing optical system 11 can be adjusted by relatively moving both of them. To adjust the focal length, the subject image can be formed on the light receiving surface of the image sensor 12 by moving the focus lens 11a.

  The zoom lens 11a and the focus lens 11b can be manually moved in the optical axis direction by an operator. Alternatively, the zoom lens 11 a and the focus lens 11 b can be moved by being driven by the zoom drive unit 16. In addition, the focus lens 11b is driven by the focus drive unit 17 so that the subject image is focused. The focus position adjustment can be automatically performed by executing an autofocus (AF) function as will be described later.

  A diaphragm 18 and a shutter 19 are disposed between the photographing optical system 11 and the image sensor 12. By adjusting the opening of the diaphragm 18, the illuminance of light incident on the light receiving surface of the image sensor 12 is adjusted. Passing and blocking of the subject light beam are switched by opening and closing the shutter 19. The aperture 18 is driven by an aperture drive unit 20 to adjust the opening. The shutter 19 is driven by a shutter drive unit 21 to be opened and closed.

  The zoom drive unit 16, the focus drive unit 17, the aperture drive unit 20, and the shutter drive unit 21 are connected to the DSP 14, and their operations are controlled by the DSP 14.

  The image sensor 12 is connected to the DSP 14 via the AFE 13. A clock signal is output from the DSP 14 to the AFE 13. The AFE 13 generates a frame signal and an image sensor driving signal based on the clock signal. The image sensor drive signal is transmitted to the image sensor 12. The image sensor 12 generates an image signal synchronized with the frame signal based on the image sensor drive signal.

  Note that pixels are arranged in a matrix on the light receiving surface of the image sensor 12. For example, in the effective pixel region, pixels are arranged at each of the positions equally divided into m × n. A pixel signal is generated according to the amount of light received at each pixel. The image signal is formed by a plurality of pixel signals generated by a plurality of pixels in the effective pixel region.

  The generated image signal is transmitted to the AFE 13. The image signal is subjected to correlated double sampling, gain adjustment, and A / D conversion in the AFE 13 to be converted into image data. The image data is transmitted to the DSP 14.

  The DSP 14 is connected to a DRAM (Dynamic Random Access Memory) 22 which is a memory for signal processing work. The image data transmitted to the DSP 14 is temporarily stored in the DRAM 22. The DSP 14 performs predetermined data processing on the image data stored in the DRAM 22.

  A liquid crystal monitor 23 is connected to the DSP 14. Image data that has undergone predetermined data processing is transmitted to the liquid crystal monitor 23. An image corresponding to the transmitted image data can be displayed on the liquid crystal monitor 23.

  An external card interface 24 is connected to the DSP 14. When a release operation described later is executed, image data subjected to predetermined signal processing is stored in an external memory (not shown) connected to the external card interface 24.

  The DSP 14 is connected to an operation input unit 15 for inputting a command for operating the digital camera 10. The operation input unit 15 includes a release button (not shown), a multi-function cross key (not shown), a power button (not shown), and the like. In response to a command input to the operation input unit 15 by the operator, the DSP 14 causes each part of the digital camera 10 to perform a necessary operation.

  For example, when a release button is half-pressed, a first switch (not shown) is turned ON, and exposure adjustment and focus position adjustment are performed.

  In the exposure adjustment, first, the amount of received light in the effective pixel region is detected based on the image data. Next, exposure adjustment, for example, aperture opening adjustment, shutter speed adjustment, and gain adjustment in the DSP 24 are performed based on the detected amount of received light.

  In addition, the detection of the amount of received light can detect the entire effective pixel region or a partial region. It is also possible to perform weighted exposure adjustment that weights the received light amount of a part of the effective pixel region and performs exposure adjustment based on an average value with the received light amount of the surrounding regions.

  In the focus position adjustment, the focus lens 11b is driven so as to form a desired subject light beam on the light receiving surface of the image sensor 12, as will be described later.

  Further, when the release button is fully pressed, a second switch (not shown) is turned on, and the image sensor 12 is driven so as to open and close the shutter 19 and capture a still image.

  The internal configuration of the DSP 14 will be described in more detail with reference to FIG. FIG. 2 is a block diagram schematically showing the internal configuration of the DSP 14. The DSP 14 includes a front data processing unit 14p1, a rear data processing unit 14p2, a tracking unit 30 (subject tracking system), an AF adjustment unit 14a, and a control unit 14c.

  Image data is transmitted from the AFE 13 to the upstream data processing unit 14p1. Image data is stored in the DRAM 22 by the pre-stage data processing unit 14p1. The image data is subjected to predetermined data processing such as color interpolation processing, white balance processing, and luminance signal generation processing. Further, the image data that has been subjected to the predetermined data processing is transmitted to the subsequent data processing unit 14p2.

  Also in the subsequent data processing unit 14p2, predetermined data processing such as clamping processing and blanking processing is performed on the image data. Image data subjected to predetermined data processing in the subsequent data processing unit 14p2 is transmitted to the external memory via the liquid crystal monitor 23 and the external card interface 24.

  Image data is also transmitted from the pre-stage data processing unit 14p1 to the tracking unit 30 and the AF adjustment unit 14a. Based on the transmitted image data, the tracking unit 30 and the AF adjustment unit 14a determine the drive position of the focus lens 11b for forming an image of a desired subject on the light receiving surface of the image sensor 12.

  More specifically, the tracking unit 30 sets some area in the entire captured image as a scan area (tracking area). The scan area is an area for including a desired subject to be focused on the light receiving surface. When a tracking target object that is a desired subject moves in a captured image, it is possible to track the tracking target object by continuously setting the moving area as a scan area.

  The AF adjustment unit 14a obtains the drive position of the focus lens 11b for forming an image of the subject included in the scan area on the light receiving surface. The drive position of the focus lens 11b is based on a contrast method, that is, a method of detecting the contrast of the scan area while moving the focus lens and obtaining the focus lens position that maximizes the contrast as the focus lens drive position. Executed.

  The driving position of the focus lens 11b may be obtained by any conventionally known AF operation. For example, it may be combined with a configuration in which focus detection is performed by a known phase difference method.

  The digital camera 10 is provided with a fixed AF function that focuses a subject at a specific position in the entire captured image and a tracking AF function that focuses a moving subject in the captured image. Which AF function is to be executed can be selected by a command input to the operation input unit 15.

  An input signal corresponding to a command input is transmitted from the operation input unit 15 to the control unit 14c. In accordance with the received input signal, the control unit 14c controls the operation of the front-stage data processing unit 14p1, the rear-stage data processing unit 14p2, the tracking unit 30, and the AF adjustment unit 14a and the respective parts of the digital camera 10. The

  For example, when performing the exposure adjustment described above, the driving of the diaphragm 18 by the diaphragm driving unit 20 is controlled by the control unit 14c. Further, when the shutter 19 is opened and closed, the opening and closing of the shutter 19 by the shutter driving unit 21 is controlled by the control unit 14c.

  Further, when the focus position adjustment is performed, the driving of the focus lens 11b by the focus driving unit 17 is controlled by the control unit 14c. A lens position signal corresponding to the drive position of the focus lens 11b obtained by the AF adjustment unit 14a is transmitted to the control unit 14c. The control of the focus driving unit 17 by the control unit 14c is executed based on the lens position signal.

  Next, the configuration and operation of the tracking unit 30 will be described in detail with reference to FIGS. FIG. 3 is a block diagram schematically showing the internal configuration of the tracking unit 30. The tracking unit 30 includes a pixel block calculation unit 31, a scan region initial setting unit 32, a candidate region setting unit 33, an image recognition unit 34, and a scan region setting unit 35. In addition, operation | movement of each site | part is controlled by the control part 14c.

  When the tracking AF function is executed, the luminance of each pixel in the effective pixel area is input to the pixel block calculation unit 31 from the previous data processing unit 14p1 as data. The pixel block calculation unit 31 calculates the luminance of the pixel block small or the pixel block large in order to facilitate the tracking process.

  As shown in FIG. 4, the small pixel block 12b is an area obtained by equally dividing the effective pixel area AA into, for example, 20 × 20, and is formed by a plurality of adjacent pixels 12p, for example, 10 × 10 pixels 12p. Is done. The luminance of the pixel block small 12b is obtained by calculating the average value of the luminances of the pixels 12p forming the pixel block small 12b. The luminance of the small pixel block 12b is sent to the image recognition unit 34 as data.

  As shown in FIG. 5, the large pixel block 12B is an area obtained by equally dividing the effective pixel area AA into, for example, 10 × 10, and is formed by a plurality of adjacent pixels 12p, for example, 20 × 20 pixels 12p. Is done. The luminance of the pixel block size 12B is obtained by calculating the average value of the luminance values of the pixels 12p forming the pixel block size 12B. The luminance of the pixel block size 12B is sent as data to the image recognition unit 34.

  Note that the amount of received light in the pixel block small 12b or the pixel block large 12B is also used to detect the amount of received light in a part of the effective pixel area AA in the exposure adjustment described above. That is, the received light amount is detected using the pixel block small 12b or the pixel block large 12B as a partial region, and photometry such as spot photometry or partial weight photometry is performed.

  When executing the AF function, the scan area initial setting unit 32 determines the position of the scan area so that the center of the effective pixel area AA of the image sensor 12 overlaps the center of the scan area. Furthermore, the size of the scan area is determined based on the luminance of the entire effective pixel area. By setting the position and size of the center, the scan area is initially set.

  The brightness of the entire effective pixel area AA is sent to the scan area initial setting section 32 as data from the preceding data processing section 14p1. When the luminance of the entire effective pixel area AA is larger than the first threshold value, the size of the scan area SA is determined to be the size of 32 pixel block small 12b which is a power of 2. Further, as shown in FIG. 6, the shape is determined to be a shape excluding the pixel block small 12b at the four corners of the rectangle formed by the pixel block small 12b of 6 rows and 6 columns.

  On the other hand, when the luminance of the entire effective pixel area AA is smaller than the first threshold value, the size of the scan area SA is determined to be the size of 32 pixel blocks large 12B, which is 2 5. Further, as shown in FIG. 7, the shape is determined to be a shape excluding the pixel block size 12B at the four corners of the rectangle formed by the 6 × 6 pixel block size 12B.

  Note that the effective pixel area AA has a boundary line between two small pixel blocks 12b or large pixel blocks 12B extending in the vertical and horizontal directions (see FIG. 6 and FIG. 7). The scan area SA can be set at an arbitrary position. The scan area SA is set by inputting a command to the operation input unit 15.

  The initially set scan area SA is transmitted to the candidate area setting unit 33 as data. In the candidate area setting unit 33, an area having the same shape as the scan area SA moved in the surrounding eight directions from the scan area SA is set as the candidate area.

  The first to eighth directions are defined as directions from the scan area for setting as a candidate area. 6 and 7, the four directions of up, down, left, and right are defined as the first, fifth, third, and seventh directions, respectively. Further, the upper left direction, the lower left direction, the lower right direction, and the upper right direction are defined as the second, fourth, sixth, and eighth directions, respectively.

  The moving distance from the scan area SA of each of the first to eighth candidate areas can be changed according to the focal length. As the focal length becomes longer, the moving distance becomes longer. For example, when the focal length is minimum, the moving distance is determined to be one pixel block small 12b or one pixel block large 12B.

  Hereinafter, the positions of the first to eighth candidate areas CA1 to CA8 when the scan area SA is formed by 32 pixel block small 12b will be exemplified. Note that the positions of the first to eighth candidate areas CA1 to CA8 are similarly determined when the scan area SA is formed by 32 large pixel blocks 12B.

  A region moved by one small pixel block 12b in the first direction from the scan region SA is set as the first candidate region CA1 (see FIG. 8). A region moved by one small pixel block 12b in the second direction from the scan region SA is set as the second candidate region CA2 (see FIG. 9). A region moved by one small pixel block 12b in the third direction from the scan region SA is set as the third candidate region CA3 (see FIG. 10). An area moved by one small pixel block 12b in the fourth direction from the scan area SA is set as the fourth candidate area CA4 (see FIG. 11). A region moved by one small pixel block 12b in the fifth direction from the scan region SA is set as the fifth candidate region CA5 (see FIG. 12). A region moved by one small pixel block 12b in the sixth direction from the scan region SA is set as a sixth candidate region CA6 (see FIG. 13). A region moved by one small pixel block 12b in the seventh direction from the scan region SA is set as a seventh candidate region CA7 (see FIG. 14). A region moved by one small pixel block 12b in the eighth direction from the scan region SA is set as the eighth candidate region CA8 (see FIG. 15).

  As the focal length becomes longer from the minimum value, the number of small pixel block 12b or large pixel block 12B, which is the moving distance, is determined to be 2, 3, 4,... And determined in the first to eighth directions. First to eighth candidate areas CA1 to CA8 are set at the positions of the movement distances.

  The set first to eighth candidate areas CA1 to CA8 are transmitted to the image recognition unit 34 as data. Further, the scan area SA initially set in the scan area initial setting unit 32 is also transmitted as data to the image recognition unit 34. Further, image data is sent from the AFE 13 to the image recognition unit 34.

  The image recognition unit 34 extracts data components corresponding to images in the scan area SA and the first to eighth candidate areas CA1 to CA8 from the image data of each frame. The extracted data component corresponds to the luminance of the pixel block small 12b or the pixel block large 12B forming each region.

  For example, in the scan area SA in the image data sent at the first timing, as shown in FIG. 16, 120, 30, 60, 55, 70, 120, 30, 60, 55, 70, 110, 100, 70, 40, 105, 40, 85, 95, 65, 25, 40, 150, 120, 60, 30, 25, 45, 100, 120, 110, 95, 80, 50, 90, 75, 80 and 20 are extracted as luminance. Note that the luminance in the pixel block size 12B is similarly extracted when the scan area SA is formed by 32 pixel block sizes 12B.

  Based on the extracted data components, the luminance of each pixel block small 12b or pixel block large 12B in the scan area SA and the first to eighth candidate areas CA1 to CA8 is binarized. That is, the average value of the luminance in the plurality of small pixel blocks 12b or the large pixel block 12B included in each region is obtained, and the luminance of the small pixel block 12b or the large pixel block 12B that is larger than the average value is converted to 1. The luminance of the pixel block small 12b or the pixel block large 12B having a luminance smaller than the average value is converted to zero.

  For example, the average value of the brightness of the scan area shown in FIG. 16 is 73.75, and the brightness of each pixel block small 12b is binarized, as shown in FIG. 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, in order of small 12b or large pixel block 12B Converted to 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0. When the scan area SA is formed by 32 large pixel blocks 12B, the binarization of the luminance in the large pixel block 12B is performed in the same manner.

  The binarized luminance of each pixel block small 12b or pixel block large 12B is sent as data to the scan region setting unit 35 for each region. In the scan area setting unit 35, it is estimated which of the first to eighth candidate areas CA1 to CA8 has moved in the image data generated by imaging at the next timing of the subject included in the scan area SA.

  This estimation is based on the binarized luminance of each pixel block small 12b or large pixel block large 12B in the scan area SA and each of the first to eighth candidate areas CA1 to CA8 of the image data at the next timing. This is performed based on the binarized luminance of the pixel block small 12b or the pixel block large 12B.

  Selection from the first to eighth candidate areas CA1 to CA8 is performed by calculation of the first to eighth determination values and determination based on the first to eighth determination values. Each operation will be described in detail below.

  The first to eighth determination values are calculated values indicating the coincidence between the image of the scan area SA and each of the first to eighth candidate areas CA1 to CA8. The first to eighth determination values are binarized in the pixel block small 12b or the pixel block large 12B that are relatively in the same position in the scan area SA and in the first to eighth candidate areas CA1 to CA8. It is calculated by comparing the luminance and counting the number of pixel block small 12b or pixel block large 12B that do not match. Therefore, it is estimated that the lower the first to eighth determination values, the higher the consistency with the image in the scan area SA.

  The scan area setting unit 35 has an EXOR circuit (not shown), and the binarized luminance in both the pixel block small 12b or both pixel block large 12B is input to the EXOR circuit. 0 is returned when the binarized luminances in both the pixel block small 12b or the both pixel block large 12B match, and 1 is returned when the binarized luminances are different.

  For example, the binarized luminance in the first candidate area CA1 has a value as shown in FIG. When the binarized luminance is input to the EXOR circuit for all the pixel block small 12b having the same relative position in the scan area SA and the first candidate area CA1, the left to right and the top to bottom 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, The values 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0 are output. In the comparison between the scan area SA and the first candidate area CA1, the number of times the value of 1 is output from the EXOR circuit is counted as the first determination value U (exor).

  Similarly, the scan area SA and the second candidate area CA2 are compared, and the number of times a value of 1 is output from the EXOR circuit is counted as the second determination value UL (exor). Similarly, the scan area SA is compared with the third candidate area CA3, and the number of times the value of 1 is output from the EXOR circuit is counted as the third determination value L (exor). Similarly, the scan area SA is compared with the fourth candidate area CA4, and the number of times the value of 1 is output from the EXOR circuit is counted as the fourth determination value DL (exor). Similarly, the scan area SA and the fifth candidate area CA5 are compared, and counted as the fifth determination value D (exor) outside the number of times when a value of 1 is output from the EXOR circuit. Similarly, the scan area SA is compared with the sixth candidate area CA6, and the number of times the value of 1 is output from the EXOR circuit is counted as the sixth determination value DR (exor). Similarly, the scan area SA is compared with the seventh candidate area CA7, and the number of times that a value of 1 is output from the EXOR circuit is counted as the seventh determination value R (exor). Similarly, the scan area SA is compared with the eighth candidate area CA8, and the number of times the value of 1 is output from the EXOR circuit is counted as the eighth determination value UR (exor).

  Minimum values of the first to eighth determination values U (exor), UL (exor), L (exor), DL (exor), D (exor), DR (exor), R (exor), UR (exor) Is determined. The candidate area corresponding to the determination value determined to be the minimum value is determined to be the area where the subject has moved and is selected. The selected candidate area is reset as the scan area SA.

  When the fixed AF function is executed, only the scan region initial setting unit 32 is driven in the tracking unit 30, and the pixel block calculation unit 31, the candidate region setting unit 33, the image recognition unit 34, and the scan region setting unit 35 are driven. Is stopped.

  The scan area SA set by driving the scan area initial setting section 32 is sent as data to the AF adjustment section 23a via the image recognition section 34 and the scan area setting section 35. Note that, unlike the tracking AF function, the scan area SA is fixed to the scan area SA set first.

  Next, the process performed for setting the scan area SA performed in the tracking unit 30 when the tracking AF function is executed will be described with reference to the flowcharts of FIGS. 19 and 20. FIG. 19 is a first flowchart for explaining the process performed for setting the scan area SA. FIG. 20 is a second flowchart for explaining the process performed for setting the scan area SA.

  This process is started when the tracking AF function is executed, that is, when the release button is half-pressed after inputting a command for switching to the tracking AF function execution. This process is repeated until the digital camera 10 is turned off or the tracking AF function is canceled.

  In step S100, the initial position of the scan area SA is set. The position of the scan area SA is set with the point C within the frame of the captured image determined by the command input by the operator as the center C.

  In the next step S101, one frame of image data is received, and the process proceeds to the next step S102. In step S102, photometry is performed. When performing photometry, first, the received light amount at each pixel 12p is sent from the preceding data processing unit 14p1 to the scan region initial setting unit 32. Based on the amount of light received by each pixel 12p in one frame of image data, the scan area initial setting unit 32 calculates the luminance of the entire effective pixel area AA.

  When the luminance of the entire effective pixel area AA is obtained, the process proceeds to step S103. In step S103, the luminance obtained in step S102 is compared with a first threshold value.

  When the luminance of the entire effective pixel area AA is larger than the first threshold value, the process proceeds to step S104. In step S104, the pixel block flag PB is set to zero. When the luminance of the entire effective pixel area AA is smaller than the first threshold value, the process proceeds to step S105. In step S105, the pixel block flag PB is set to 1.

  After step S104 or step S105 ends, the process proceeds to step S106. In step S106, a scan area SA corresponding to the position set in step S100 and the pixel block set in step S104 or step S105 is set.

  In the next step S107, the focal length of the photographing optical system 11 is detected. First, the relative position between the zoom lens 11a and the focus lens 11b is detected by the zoom drive unit 16 as a zoom stage. The detected zoom level is transmitted as data to the candidate area setting unit 33 via the control unit 14c. The candidate area setting unit 33 detects the focal length corresponding to the zoom stage.

  When the focal length is detected, the moving distance from the scan area to the candidate area is determined according to the focal distance in the next step S108. When the movement distance is determined, the first to eighth candidate areas are set according to the pixel block and the movement distance set in step S104 or step S105.

  After the setting of the first to eighth candidate areas CA1 to CA8 is completed, the process proceeds to step S109. In step S109, it is determined whether the pixel block flag PB is 0 or 1.

  When the pixel block flag PB is 0, the process proceeds to step S110, where the luminance of the pixel block small 12b in the scan area SA is binarized. When the pixel block flag PB is 1, the process proceeds to step S111, and the luminance of the pixel block size 12B in the scan area SA is binarized.

  It progresses to step S112 after completion | finish of step S110 or step S111. In step S112, one frame of image data generated at the next timing is received. After receiving the image data, the process proceeds to step S113, where it is determined whether the pixel block flag PB is 0 or 1.

  When the pixel block flag PB is 0, the process proceeds to step S114, and the luminance of the pixel block small 12b in the first to eighth candidate areas CA1 to CA8 is binarized. When the pixel block flag PB is 1, the process proceeds to step S115, and the luminance of the pixel block size 12B in the first to eighth candidate areas CA1 to CA8 is binarized.

  It progresses to step S116 after completion | finish of step S114 or step S115. In step S116, the first to eighth determination values U (exor) based on the binarized luminance in the scan area SA and the binarized luminance in the first to eighth candidate areas CA1 to CA8. , UL (exor), L (exor), DL (exor), D (exor), DR (exor), R (exor), and UR (exor) are calculated.

  When the first to eighth determination values U (exor), UL (exor), L (exor), DL (exor), D (exor), DR (exor), R (exor), and UR (exor) are obtained. The process proceeds to step S117. In step S117, the first to eighth determination values U (exor), UL (exor), L (exor), DL (exor), D (exor), DR (exor), R (exor), UR (exor) ) Is determined which is the minimum value. A candidate area corresponding to the determination value having the smallest determination value is set as the position of the new scan area SA.

  When step S117 is completed, the process returns to step S102, and the processes of step S102 to step S117 are repeated thereafter. Therefore, the scan area SA is reset by repeating the processes of step S117 and steps S102 to S106.

  According to the pattern matching system of the present embodiment having the above-described configuration, it is possible to effectively use the digital arithmetic unit used in the pattern matching system. The number of bits handled by the digital computing unit is a power of 2, and the number of bits handled and the number of bits necessary for calculating the luminance of the pixel block small 12b or the pixel block large 12B of 2 to the power of 2, and binarization, and calculating the determination value This is because it is possible to match the number of bits handled by the arithmetic unit.

  Further, according to the pattern matching system of the present embodiment, the luminance of each pixel block is binarized, so that the stability of pattern matching can be improved. For example, when a light source that generates flicker, such as a fluorescent lamp, is received by a scan area or a candidate area, local consistency may be low even if the overall consistency is high. Since the inconsistency is reduced, the stability is improved.

  Further, according to the pattern matching system of the present embodiment, since the size of the pixel block used for pattern matching is switched according to the brightness of the entire effective pixel area AA, it is possible to reduce erroneous determination due to noise. Become. When the total amount of received light is low, noise with respect to the signal intensity of the pixel signal increases. However, since the number of pixels used for obtaining the luminance of the pixel block is increased by increasing the size of the pixel block, the influence of noise is neutralized in the entire pixel block.

  Further, according to the subject tracking system of the present embodiment, the movement distance from the scan area SA to the first to eighth candidate areas CA1 to CA8 is changed according to the focal length of the photographing optical system 11, so It is possible to improve accuracy. This is because the moving speed of the same moving subject on the image increases as the focal length increases.

  Further, according to the subject tracking system of the present embodiment, the luminance in the pixel block small 12b or the pixel block large 12B is used for the area for detecting the received light amount in spot metering, partial weight metering, etc. It is possible to increase the processing speed as compared with the case of performing.

  In this embodiment, the number of small pixel blocks 12b or large pixel blocks 12B that form the scan area SA or candidate area is 32, which is the fifth power of 2, but is handled by a digital computing unit used in the pattern matching system. The number of pixel blocks may be a power of 2 that is the same as the number of possible bits.

  In the present embodiment, the configuration is such that the moving direction of the subject is one of the eight directions of the first to eighth directions. However, any configuration may be used as long as it is determined from a plurality of directions.

  In the present embodiment, when the zoom lens is at the wide end, the moving distance from the scan area SA to the first to eighth candidate areas CA1 to CA8 is one pixel block small 12b or one pixel block large 12B. However, the number of pixel block small 12b or pixel block large 12B to be moved may be any number. An effect similar to that of the present embodiment can be obtained as long as the moving distance is changed in accordance with the focal length.

  Further, in the present embodiment, the shapes of the scan area SA and the first to eighth candidate areas CA1 to CA8 are cross-shaped, but any shape may be used.

  In the present embodiment, the luminance of the pixel block 12b in the scan area SA and the first to eighth candidate areas CA1 to CA8 is binarized. It may be hierarchized. Furthermore, it does not have to be hierarchized. However, the above-described effects can be obtained by binarization or hierarchization.

  Alternatively, in the present embodiment, when the luminance of the pixel blocks 12b and 12B is hierarchized by three or more, or when hierarchization is not performed, the absolute difference in luminance between the pixel block at the same position in the scan area SA and each candidate area A configuration may be used in which each determination value is calculated by returning 0 when the value is equal to or less than a predetermined reference value. In the case of binarized luminance as in the present embodiment, this corresponds to setting the reference value to 0.

  Further, in the present embodiment, the size of the pixel block used for pattern matching is switched in two stages according to the brightness of the entire effective pixel area AA, but it may be switched in any number of stages. Alternatively, it is possible to exhibit the effect of effectively using the digital arithmetic unit without switching. However, the above-described effects can be obtained by switching according to the brightness.

  In this embodiment, the size of the pixel block used for pattern matching is switched according to the brightness of the entire effective pixel area AA, but the configuration is switched according to the brightness of a part of the effective pixel area AA. May be.

  In the present embodiment, the moving distance from the scan area SA to the first to eighth candidate areas CA1 to CA8 is changed according to the focal length of the photographing optical system. It is possible to demonstrate the effect of effective use of the vessel. However, the effects as described above can be obtained by changing the movement distance according to the focal length.

  In the present embodiment, the pixel block is also used as an area for detecting the amount of received light in spot metering, partial weight metering, etc., but the area for detecting the amount of received light in order to calculate white balance The structure used also as may be sufficient. Alternatively, it is possible to exhibit the effect of effectively using the digital arithmetic unit without using it. However, when the calculation is performed based on the amount of received light for each of the areas, it is possible to increase the overall processing speed by sharing the luminance of the pixel block.

  In the present embodiment, the EXOR circuit is used to determine whether or not the binarized luminances of the pixel block small 12b in the scan area SA and the first to eighth candidate areas CA1 to CA8 match. However, the determination may be made using, for example, an EXNOR circuit. Moreover, you may determine whether it corresponds by any other means.

  In the present embodiment, the determination value indicating the coincidence between the scan area SA and the first to eighth candidate areas CA1 to CA8 is calculated based on the luminance in the pixel blocks 12b and 12B. For example, the determination value may be calculated based on the signal intensity of the green pixel signal output from the pixel covered with the G color filter. Further, the determination value may be calculated based on the signal intensity of a pixel signal that is another color signal.

  In the present embodiment, the pixels are arranged in a matrix in the effective pixel area AA of the image sensor 12, but it is sufficient that the pixels are arranged in a two-dimensional manner.

  In the present embodiment, the tracking unit 30 tracks a desired subject, and the AF function is executed on the tracked subject. However, the tracking function by the tracking unit 30 is applied to other functions. Is possible. For example, a monitoring camera that tracks a desired subject and displays that the subject is being tracked on the monitor, a camera that tracks a moving subject, and performs automatic exposure correction on the tracked subject. Is also applicable.

  In the present embodiment, the pattern matching is used for tracking the subject, but can be applied to, for example, a face authentication system.

1 is a block diagram schematically showing an internal configuration of a digital camera having a subject tracking system that tracks a subject by a pattern matching system to which an embodiment of the present invention is applied. FIG. It is a block diagram which shows schematically the internal structure of DSP. It is a block diagram which shows roughly the internal structure of a tracking part. It is a figure which shows the structure of a pixel block small. It is a figure which shows the structure of a pixel block size. It is a figure for demonstrating the shape of the scanning area | region formed by pixel block small. It is a figure for demonstrating the shape of the scanning area | region formed by pixel block size. It is a figure which shows the positional relationship of the 1st candidate area | region with respect to a scanning area | region. It is a figure which shows the positional relationship of the 2nd candidate area | region with respect to a scanning area | region. It is a figure which shows the positional relationship of the 3rd candidate area | region with respect to a scanning area | region. It is a figure which shows the positional relationship of the 4th candidate area | region with respect to a scanning area | region. It is a figure which shows the positional relationship of the 5th candidate area | region with respect to a scanning area | region. It is a figure which shows the positional relationship of the 6th candidate area | region with respect to a scanning area | region. It is a figure which shows the positional relationship of the 7th candidate area | region with respect to a scanning area | region. It is a figure which shows the positional relationship of the 8th candidate area | region with respect to a scanning area | region. It is a figure which shows an example of the brightness | luminance for every pixel block small in a scanning area | region. It is a figure which shows the binarized brightness | luminance about the pixel block small of the scanning area | region shown in FIG. It is a figure which shows an example of the binarized brightness | luminance for every small pixel block in a 1st candidate area | region. It is a 1st flowchart for demonstrating the process performed for the setting of a scanning area | region. It is a 2nd flowchart for demonstrating the process performed for the setting of a scanning area | region.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Digital camera 11a Variable magnification lens 11b Focus lens 12b, 12B Small pixel block, Large pixel block 12p Pixel 14 DSP (Digital Signal Processor)
14p1 Pre-stage data processing unit 16 Zoom drive unit 17 Focus drive unit 30 Tracking unit 31 Pixel block calculation unit 32 Scan region initial setting unit 33 Candidate region setting unit 34 Image recognition unit 35 Scan region setting unit AA Effective pixel region C Center of scan region CA1 to CA8 1st to 8th candidate area SA scan area

Claims (6)

A pattern matching system that outputs a determination value for indicating a high degree of coincidence between a first image and a second image,
The first and second images corresponding to the first and second images are formed by region signals whose signal intensity changes according to the luminance of each of the power-of-two pattern regions that form the first and second images. An acquisition unit for acquiring the image data as digital data;
A comparison unit for comparing signal strengths of the region signals of the pattern region at the same position in the first and second images;
A determination unit that calculates the determination value that varies according to the number of the pattern regions in which the absolute value of the signal intensity difference between the region signals is equal to or less than a predetermined reference value in the comparison by the comparison unit; and outputs the determination value and an output unit that,
The pattern matching system , wherein when the luminance of the first image or the whole of the second image is lower than a first threshold, the size of the pattern region is widened .   A binarization unit that binarizes signal intensity of the region signal in the first and second image data, and the comparison unit includes the pattern region at the same position in the first and second images; The pattern matching system according to claim 1, wherein the binarized signal intensities of region signals are compared. The acquisition unit receives a pixel signal whose signal intensity changes according to the luminance of each of the plurality of pixels forming the pattern region, and the pixel signal of the plurality of pixels forming the single pattern region. An area signal generation unit for generating the area signal based on
When the luminance of the entire first image or the second image is lower than the first threshold, the acquisition unit responds to the luminance of the entire large pattern area formed by the enlarged pattern area. Generate large area signals with varying signal strength,
When the luminance of the first image or the entire second image is lower than the first threshold, the comparison unit is configured to have the large area of the large pattern area at the same position in the first and second images. Compare the signal strength of the signals,
The said determination part calculates the said determination value which changes according to the number of the said large pattern area | region where the absolute value of the signal strength difference of the said large area signal becomes below the said reference value. Item 3. The pattern matching system according to Item 2. A subject tracking system that tracks the movement of a tracking target that is a specific subject in continuously captured images,
An initial setting unit that initially sets a predetermined part of the captured image as a tracking region for tracking the tracking object;
A candidate area setting unit for setting first and second candidate areas that are areas moved in the first and second directions from the tracking area;
A reference image that is an image of the tracking area in the captured image captured at the first timing is detected, and the first and second images in the captured image captured at a second timing after the first timing are detected. An image detection unit for detecting first and second candidate images that are images of two candidate areas;
Based on the reference image and the first and second candidate images, the signal intensity changes according to the luminance of each of the power-of-two pattern regions forming the reference image and the first and second candidate images. An area signal generator for generating an area signal;
A comparison unit that compares the signal intensity of the region signal of the pattern region at the same position in the first candidate image or the second candidate image with the reference image;
A first determination unit that calculates a determination value that changes according to the number of pattern regions in which the absolute value of the signal intensity difference between the region signals is equal to or less than a predetermined reference value in the comparison by the comparison unit;
Based on the determination value for the reference image and the first image or the second image, the moving direction of the tracking target object from the first timing to the second timing is changed to the first and second. A second determination unit that determines which of the two directions;
The second determination unit includes a resetting unit that resets a candidate area that has moved in the direction determined as the movement direction of the tracking target object as a tracking area,
When the brightness of the reference image, the first candidate image, or the entire second candidate image is lower than a first threshold, the size of the pattern area is increased.
Subject tracking system. The candidate area setting unit is configured to change the first and second from the tracking area according to a focal length of an imaging optical system for forming an optical image of the tracking target object on a light receiving surface of an imaging element. The subject tracking system according to claim 4, wherein the moving distance to the candidate area is set to be long . An exposure adjustment system capable of adjusting the exposure when the optical image of the tracking object is received by the image sensor in accordance with the luminance of a part of the optical image incident on the image sensor;
The subject tracking system according to claim 4 or 5 , wherein the brightness of the pattern area is used as a brightness of a part of the optical image for adjusting the exposure amount .

Images