JP2001119622A - Image-puckup device and control method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To effectively conduct automatic focus control, automatic exposure control, automatic white balance control and electronic zoom control, even if an object is not positioned at the center of a finder. SOLUTION: This image-pickup device, where a CCD (102) converts an optical image obtained via an image forming optical system (101) into an electric image signal and it is stored, is provided with a read-only memory (108) that stores a specific image pattern, a correlation value calculation section (109) that calculates a correlation value between a partial image of the stored image signal and the specific image pattern, a data read control circuit (111) that detects a maximum value of the correlation value calculated by the correlation value calculation section, and an AF, AE, AWB control circuit (113), that conducts prescribed control on the basis of the partial image corresponding to the maximum value detected.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video camera device.

[0002]

2. Description of the Related Art Recent consumer video cameras are equipped with various automatic functions and additional functions. For example, an automatic focus adjustment (AF), an automatic exposure adjustment (AE), an automatic white balance adjustment (AWB) mechanism, an electronic zoom function, and the like are typical examples. Conventionally, AF, AE, and AWB use a system that obtains a control evaluation value from a captured image signal without using an external detection mechanism such as an infrared sensor or a light amount sensor, and focuses on information at the center of the captured image. In addition, various evaluation values are often generated by adding peripheral information in addition thereto. In the case of electronic zoom, the center of the screen is almost always enlarged. This is because the subject you want to shoot is
This is because it is assumed that it is often located at the center of the screen.

In addition, in recent years, in the field of image processing apparatuses, apart from the above-mentioned consumer video camera technology, the position of an object having a specific shape in an image is determined by correlation matching with template data representing a specific shape. An image recognition technique using a template matching method for estimating has been proposed.

[0004]

However, in the above prior art, it is assumed that the position of the subject to be photographed is near the center of the screen. Has to sacrifice the degree of freedom of the screen configuration at the time of shooting. In addition, there is a problem that only the central portion of the captured image can be enlarged even with the electronic zoom.

Further, when image recognition by the template matching method is used for a natural image, there is a problem that a desired subject is not always shown even if the correlation between the template data and the input image is high.
To deal with this problem, for example, in Japanese Patent Laid-Open No. 05-28273, a maximum value of a correlation value between a pre-stored template image and an input image, or a maximum value of a correlation value between a differential image of the template image and the input image, and the like are described. By obtaining the correlation values of the image types, the detection rate of a desired subject is improved. However, they do not essentially solve this problem.

Further, in the template matching method,
There is a problem that the amount of calculation of the correlation between the template data and the input image is very large, and it is difficult to process at a speed close to the video frame rate. To solve this problem, for example, in JP-A-07-22076, a correlation value is calculated with a predetermined search width, the obtained correlation values are sorted in descending order, and the periphery of the position where the correlation value is obtained is the predetermined value. The amount of calculation is reduced by searching again with a width smaller than the search width of. However, this is intended for inspection using image information, and the amount of calculation has not been sufficiently reduced to detect the position of a specific subject in a natural image as in a consumer video camera.

[0007]

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and an optical image obtained through an image forming optical system according to the present invention is converted into an electric image signal by a photoelectric conversion element. An imaging device that stores a specific image pattern, a storage unit that stores a specific image pattern, a partial image of the stored image signal, a correlation value calculation unit that calculates a correlation value of the specific image pattern, and the correlation value calculation unit. The system includes a maximum value detection unit that detects a maximum value of the calculated correlation value, and a control unit that performs predetermined control based on a partial image corresponding to the maximum value detected by the maximum value detection unit.

[0008] Further, according to the present invention, there is provided a control method of an image pickup apparatus for converting an optical image obtained through an image forming optical system into an electric image signal by a photoelectric conversion element, storing the electric image signal, and previously storing a specific image pattern. A partial image of the stored image signal, a correlation value calculation step of calculating a correlation value of the specific image pattern, a maximum value detection step of detecting a maximum value of the correlation value calculated in the correlation value calculation step, A control step of performing predetermined control based on the partial image corresponding to the local maximum value detected in the local maximum value detecting step.

According to a preferred aspect of the present invention, the local maximum value detecting means includes a comparing means for comparing the correlation value calculated by the correlation value calculating means with a predetermined value; A first shift unit that shifts the position of the partial image of the image signal by a predetermined amount when it is determined that the image signal is lower than the predetermined value.
When the correlation value is determined to be greater than or equal to a predetermined value by the shifting means, the direction in which the correlation value changes most is calculated, and the position of the partial image of the image signal is shifted in the calculated direction. And a maximum value detecting means for detecting a maximum value of a correlation value between the partial image shifted at the position shifted by the second shifting means and the specific image pattern.

The maximum value detecting step may include a comparing step of comparing the correlation value calculated in the correlation value calculating step with a predetermined value, and a step of determining that the correlation value is lower than the predetermined value in the comparing step. A first shift step of shifting the position of the partial image of the image signal by a predetermined amount, and a direction in which the correlation value changes most when the correlation value is determined to be equal to or more than the predetermined value in the comparing step. A second shift step of shifting the position of the partial image of the image signal in the calculated direction, and a correlation value between the partial image shifted at the position shifted by the second shift step and the specific image pattern. A maximum value detecting step of detecting a maximum value.

[0011] According to the above configuration, a search based on a calculation amount operable at a video frame rate can be realized by proposing an efficient search method of the template matching method.

Further, according to a preferred aspect of the present invention, the apparatus further comprises electronic zoom means for enlarging / reducing the image signal, wherein the local maximum value detecting means comprises a maximum value detected by the maximum value detecting means. Means for setting the magnification of the electronic zoom means so that the correlation value between the partial image corresponding to the correlation value of (1) and the specific image pattern is maximized.

[0013] Further, the method further includes an electronic zoom step of enlarging / reducing the image signal, wherein the local maximum value detecting step includes the step of identifying the partial image corresponding to the correlation value of the maximum value detected in the maximum value detecting step. The method further includes a step of setting a magnification in the electronic zoom step so that a correlation value with the image pattern is maximized.

Further, according to a preferred aspect of the present invention, it is determined whether or not the partial image corresponding to the local maximum detected by the local maximum detecting means is a target object indicated by the specific image pattern. When the determination means determines that the partial image is not the target object, the control means is deactivated and the first shift means is activated, and the image signal is output. Means for shifting the position of the partial image by a predetermined amount.

A determining step of determining whether or not the partial image corresponding to the local maximum detected in the local maximum detecting step is a target object indicated by the specific image pattern;
In the determining step, when it is determined that the partial image is not the target object, the control step is deactivated, and the first shift step is activated, so that the partial image of the image signal is The method further includes the step of shifting the position by a predetermined amount.

According to the above configuration, in the template matching method for a natural image, for example, even if a strong correlation is detected with respect to a background similar to the above-described specific pattern, a case where the pattern is truly a specific pattern by performing discriminant analysis. , The final pattern detection can be performed.

According to a preferred aspect of the present invention, the predetermined control is at least one of automatic focus control, automatic exposure control, and automatic white balance control, or at least one of enlargement processing and reduction processing. including. According to the above configuration, a specific image pattern of a target object such as a human face in a captured image is detected by a template matching method, and automatic detection is performed based on a portion of the captured image corresponding to the detected specific image. Since focus control, automatic exposure control, automatic white balance control, and electronic zoom control can be performed, it is possible to effectively perform shooting with each of the above controls even if the subject is not located at the center of the screen.

Preferably, a plurality of similar image patterns having different sizes are stored as the specific image pattern, and one of the plurality of image patterns is selected.

Further, according to a preferred aspect of the present invention, the apparatus further comprises electronic zoom means for enlarging / reducing the image signal, wherein the correlation value calculating means comprises an image enlarged or reduced by the electronic zoom means. A correlation value between the partial image of the signal and the specific image pattern is calculated. The image processing apparatus further includes an electronic zoom step of enlarging / reducing the image signal, wherein the correlation value calculating step includes a correlation value between a partial image of the image signal enlarged or reduced in the electronic zoom step and the specific image pattern. Is calculated. According to the above configuration, the local maximum value can be detected more efficiently.

Further, according to a preferred aspect of the present invention, in the local maximum value detection, the image signal used for detecting a position where the correlation value has a local maximum value is blurred by a low-pass filter or the like. The image signal used when obtaining the maximum value of the correlation value when the magnification is changed using an image uses an unblurred image.

Preferably, the imaging device is a video camera.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

(First Embodiment) FIG. 1 shows a configuration of an image pickup apparatus according to a first embodiment of the present invention. In FIG. 1, reference numeral 101 denotes an imaging optical system, 102 denotes an image sensor such as a CCD,
03 is an A / D converter, 104 is a camera signal processing unit,
105 is a video signal output terminal, 106 is a random access memory (RAM), 107 is a memory interface circuit, 108 is a read-only memory (ROM) for storing a template image, 109 is a correlation value calculator, 110 is a discriminant analysis circuit, 111 Is a data read control unit, and 112 is A
F, AE, AWB evaluation value detection circuit, 113 is AF, A
E and AWB control circuits 114 are template size control circuits.

FIG. 2 shows the data read control circuit 1 shown in FIG.
It is a block diagram which shows the detailed structure of No. 11. 2, reference numeral 201 denotes a template size control input terminal for inputting a signal from the template size control circuit 114;
02 is a horizontal search width data holding unit, 203 is vertical search width data, 204 is a first switch, and 205 is a second switch.
, 206 is a third switch, 207 is a fourth switch, 208 is a first adder, 209 is a second adder, 210 is a horizontal counter register, 211 is a vertical counter register, and 212 is address data generation. Circuit, 21
3 is a frame signal generation circuit, 214 is a frame signal output terminal, 215
Is a correlation value input terminal, 216 is a correlation value threshold register, 21
7 is a comparator, 234 is an AND element, 218 is a correlation value buffer, 219 is a subtractor, 220 is a coefficient generation circuit,
221 is a horizontal correlation differential value buffer, 222 is a vertical correlation differential value buffer, 223 is a gradient direction calculation circuit, 224 is a cosine function circuit, 225 is a sine function circuit, 226 is a first multiplier, and 227 is a second multiplier. , 228 a horizontal counter increment unit, 229 a vertical counter increment unit, 230 a fifth switch, 231 a sixth switch, 232 a determination result state input terminal, 233
Is an address data output terminal.

Hereinafter, the operation of the imaging apparatus according to the first embodiment of the present invention will be described.

The imaging optical system 101 shown in FIG. 1 includes a focus adjusting mechanism and an iris adjusting mechanism, projects incident light on a light receiving surface of an image sensor 102, and the image sensor 102 converts the image signal into an image signal corresponding to the incident light. . The image sensor 102 outputs this image signal to the A / D converter 103 and converts it into a digital image signal. The digital image signal output from the A / D converter 103 is output to a camera signal processing unit 104
Receives predetermined processing and is output from the video signal output terminal 105. Note that the camera signal processing unit 104 includes a white balance adjustment mechanism. On the other hand, the camera signal processing unit 104 stores the luminance signal of the input image signal in the RAM 1
06 and store it. In the first embodiment, a target object (in the first embodiment, a human face) is detected based on the luminance signal of the image signal stored in this manner, and the obtained target object is obtained. AF, A based on the result
E and AWB control values are calculated.

The detection and calculation procedure according to the first embodiment of the present invention will be described with reference to the flowcharts of FIGS.

The ROM 108 stores a plurality of template image data indicating a desired object, such as multi-valued template image data of a human face. In step S 101, a template of a size designated by the template size control circuit 114 is stored. Data is read out.
Next, in step S102, from the luminance image data stored in the RAM 106, a partial image having the same size as the template data read from the ROM 108 is read by the memory interface circuit 107. In addition,
The partial image read out here is the data read control circuit 1
Reference numeral 11 denotes a partial image at the address of the RAM 106 designated (here, determined based on the coordinates (x, y) of the image).

Thereafter, in step S103, the correlation value calculator 109 calculates a correlation value between the template data read in step S101 and the luminance image data read in step S102. Thereafter, the data read control circuit 111 determines the RAM1 based on the correlation value.
While updating the read address from 06, a search is made for a partial image in which the correlation value shows the maximum value (steps S104 to S104).
109). The procedure for searching for the maximum value of the correlation value will be described later in detail.

The partial image corresponding to the searched local maximum value is input to the discriminant analysis circuit 110. If it is determined in step S110 that the searched partial image is a human face image, a discrimination signal is read out. Output to the circuit 111. In the next step S111, the data read control circuit 111 receives this determination signal, and based on the address of the partial image corresponding to the local maximum value held therein and the template size designated by the template size control circuit 114. A frame signal is generated and output to the AF, AE, AWB evaluation value detection circuit 112. Therefore, in step S112, the AF, AE, and AWB evaluation value detection circuit 112 detects an evaluation value for focus, aperture, and white balance control based on a partial image indicating the periphery of a human face defined by the frame signal. . The AF, AE, and AWB control circuit 113 controls a focus mechanism and an iris mechanism included in the imaging optical system 101 and controls a white balance adjustment mechanism included in the camera signal processing unit 104 based on the evaluation values.

Next, with reference to FIGS. 2 and 4, the operation of the data read control circuit 111 and the procedure for searching for the maximum value of the correlation value will be described in detail.

First, the correlation value calculated by the correlation function calculator 109 in step S103 of FIG. 3 is input to the data readout control circuit 111 from the correlation value input terminal 215, and in step S104, the correlation value threshold register 216 is output from the comparator 217 in step S104. Is compared with a threshold value (here, a predetermined value 0.2 experimentally obtained).

If the correlation value is smaller than 0.2 (NO in step S104), it is determined that no face image candidate has been detected yet because the correlation is low.
Proceed to 105. In step S105, the comparator 2
17, the third and fourth switches 206 and 207
Is set to the 0 side. The third and fourth switches 20
6 and 207 are connected to the first and second switches 20.
Horizontal search width data holding unit 2 selected in 4, 205
02 (search width Δx) and vertical search width data holding unit 20
3 (search width Δy). Each search width data 202 and 2
03 is selected according to the designation of the template size control circuit 114 input from the template size control input terminal 201, and the search width according to the assumed face image size is selected. In this way, the search widths Δx and Δy are passed through the third and fourth switches 206 and 207 and the current counter values (x,
y) (step S106), and are stored in the horizontal counter register 210 and the vertical counter register 211, respectively.

Therefore, each counter register 210, 21
1 means that the counter value is sequentially changed by a predetermined search width while the correlation value is smaller than 0.2. The counter values of the counter registers 210 and 211 are sent to the address data generation circuit 212, and in step S107, the range defined by the designation input from the template size control input terminal 201 centering on the address indicated by the counter value Creates address data to be sent to the RAM 106 and outputs it from the address data output terminal 233 to the memory interface circuit 107 (step S
107). With the address data thus output, R
The next partial image is read from AM 106, and the next correlation value is calculated.

On the other hand, when the correlation value is 0.2 or more (YES in step S104), it is determined that the face image candidate has been detected because the correlation is high, and step S1 is performed.
Proceed to 08. In step S108, the comparator 21
7, the third and fourth switches 206, 20
7 is set to 1 side. In this state, the RAM 106
A maximum value search for obtaining the maximum value of the correlation value is performed using the address of the parameter as a parameter (step S109).

In general, a certain parameter is represented by Xi (i = 0,
1, 2,..., N), and when the evaluation value is Y, the local maximum value Yp of the evaluation value can be obtained by observing the partial differential of the evaluation value Y with respect to Xi. In this method, a combination (direction) of the parameter Xi that changes the evaluation value Y most is obtained, the parameter Xi is updated by a predetermined width in that direction, and the maximum value Yp is obtained by repeating this operation. For example, when detecting the position of a human face from a two-dimensional image, the parameters are horizontal and vertical coordinates representing the position, and the evaluation value is the correlation value. Therefore,
The local maximum search in this problem can be formulated as follows.

That is, assuming that the partial derivative of the parameter with respect to the evaluation value is dx, dy, dx = γ (x + 1, y) −γ (x, y) (1) dy = γ (x, y + 1) −γ (x, y y) (2)

Here, γ (x, y) is a correlation value at the pixel coordinates (x, y).

The direction θ in which the evaluation value is changed most greatly is as follows: θ = arctan (dy / dx) (3)

Can be represented by Further, the updated values xnew and ynew of the parameters are xnew = x + A * cos θ (4) ynew = y + A * sin θ (5)

Can be represented by Note that xnew, y
new is the coordinates of the pixel to be updated, and A is a coefficient that decreases as the correlation value increases. The coefficient A is calculated by the coefficient generation circuit 220. When the correlation value is low, the coordinate is updated with a large width because the deviation from the position where the maximum value is obtained is large. When the correlation value is high, the coordinate value approaches the maximum value. In order to finely update the value, a coefficient of a function that is inversely proportional to the correlation value is generated. In the configuration of FIG. 2, equation (1) is calculated as follows. First, the current correlation value (γ ₀ ) is stored in the correlation value buffer 218.
(X ₀ , y ₀ )) (step S20 in FIG. 4).
1). This is a correlation value for the current coordinates held in each counter register 210, 211.

Next, the fifth switch 230 is switched, and a predetermined value (+1 in this case) is supplied from the horizontal counter increment unit 228 through the third switch 206 to the first switch.
Of the horizontal counter register 21
The value of 0 is incremented by one. As a result, a correlation value at a position shifted by one in the horizontal direction with respect to the current coordinates is calculated (step S202), and is input from the correlation value input terminal 215. Therefore, the subtractor 219 has γ ₀ (x ₀ +
(1, y ₀ ) and γ ₀ (x ₀ , y ₀ ) are input, and dx in Expression (1) is calculated and held in the horizontal correlation differential value buffer 221 (step S203).

Similarly, the sixth switch 231 is switched to apply a predetermined value (here, +1) from the vertical counter increment unit 229 to the second adder 209 via the fourth switch 207, and the vertical counter register 2
The value of 11 is incremented by one. As a result, a correlation value at a position shifted by one in the vertical direction with respect to the current coordinates is calculated (step S204), and is input from the correlation value input terminal 215. As a result, dy of the equation (2) is calculated in the subtractor 219.
Is calculated and stored in the vertical correlation differential value buffer 222 (step S205).

Using the dx and dy obtained in this way, the calculation of the equation (3) is performed by the gradient direction calculation circuit 223 to obtain the direction θ (step S206), and then the calculation of the equation (4) is performed by the coefficient generation circuit. 220, a cosine function circuit 224, a first multiplier 226, and a third adder 208. Also,
The calculation of equation (5) is performed using the coefficient generation circuit 220 and the sine function circuit 2
25, the second multiplier 227 and the fourth adder 209 (step S207).

Thus, each counter register 210,
211 will indicate xnew, a ynew of formula (4) and (5), the coordinates xnew, ynew (i.e., _x 1, _{y 1} after update) calculate the correlation values at (step S208).

In step S209, step S2
01 and the correlation value obtained in step S208, and when the correlation value γ ₀ at the updated position is lower than the correlation value γ _{1 at} the position before updating, the maximum correlation at the position before updating is obtained. It is determined that the value is a value, and the position is stored (step S210).

As described above, by repeatedly performing the calculations of the above equations (1) to (5), the position where the correlation value takes the maximum value is detected.

The signal input from the discrimination result state input terminal 232 is a signal representing the discrimination result in the discrimination analysis circuit 110, and the partial image corresponding to the detected maximum value is a human face image. (Fig. 3
The determination result state signal, the template size control input 201, and the values of the counter registers 210 and 211 are sent to a frame signal generation circuit 213, where a frame signal is generated and output. (Step S112).

(Second Embodiment) Next, a second embodiment will be described.

FIG. 5A is a block diagram showing a configuration of an image pickup apparatus according to the second embodiment of the present invention. FIG.
In (a), 301 is a CCD image sensor, and 302 is A /
D converter 303, camera signal processing unit, 304, random access memory (RAM), 305, memory interface circuit, 306, electronic zoom processing unit, 307, video output processing unit, 308, read-only memory (RO)
M) and 309 are correlation value calculation units, 310 is a discriminant analysis circuit,
Reference numeral 311 denotes a read address control unit, and 312 denotes a video signal output terminal.

FIG. 5B is a diagram showing an example of the configuration of the correlation value calculator 309 shown in FIG. 5A. FIG.
In (b), 320 is a low-pass filter (LP
F) 322 is a correlation value calculation circuit, 321 is a RAM 304 input via a memory interface circuit 305
Is a switch for switching between inputting the image data of (1) directly to the correlation value calculation circuit 322 or inputting the image data via the LPF 320. The switch 321 is switched based on an output from a zoom magnification control circuit 413 described later with reference to FIG. This switching operation will be described later.

FIG. 6 is a block diagram showing a detailed configuration of the read address control unit 311 shown in FIG. 6, reference numeral 401 denotes a template size control unit, 402 denotes a horizontal search width data holding unit, 403 denotes a vertical search width data holding unit, 404 denotes a first switch, 405 denotes a second switch, and 406 denotes a third switch. , 407 are a fourth switch, 408 is a first adder, 409 is a second adder, 410 is a horizontal counter register, 411 is a vertical counter register, 412 is an address data generation circuit, 41
3 is a zoom magnification control circuit, 414 is a zoom control signal output terminal, 415 is a correlation value input terminal, 416 is a correlation value threshold register, 417 is a comparator, 418 is a correlation value buffer, 419 is a subtractor, 420 is a coefficient generation circuit, 421 is a horizontal correlation differential value buffer, 422 is a vertical correlation differential value buffer 222, 423 is a gradient direction calculation circuit, 424 is a cosine function circuit, 425 is a sine function circuit, 426 is a first multiplier, and 427 is a second multiplication. , 428 is a horizontal counter increment value, 429 is a vertical counter increment value, 430 is a fifth switch, 431 is a sixth switch, 432 is a determination result state input, 433 is an address data output, and 434 is a first AND element. 435, a maximum correlation value detection circuit; 436, a local maximum value determination circuit; 437, a second AND element; Denotes a third AND element.

Hereinafter, the operation of the imaging apparatus according to the second embodiment of the present invention will be described.

An image signal obtained by the CCD image pickup device 301 is digitized by an A / D converter 302, processed by a camera signal processing unit 303, converted into a video signal, and converted into a video signal by a memory interface circuit 305.
It is temporarily stored in the AM 304. The RAM 304 has a capacity capable of storing image data of a plurality of frames.
The image from the camera signal processing unit 303 is stored at a first address fixed in advance. Electronic zoom processing unit 3
06 reads out the output image of the camera signal processing unit stored at the first address of the RAM 304, and enlarges or reduces the image. The read address and magnification at this time are
The initial value is given by the read address control unit 311. The enlarged or reduced image data is stored in the RAM 30
4 is stored in a second fixed address. The video output processing unit 307 reads out the image data stored in the RAM 304 through the memory interface circuit 305, and outputs a video signal from the video signal output terminal 312 in a form according to the video signal format. At this time, the output image of the camera signal processing unit is output as it is if the first address of the RAM 304 is matched, and the image subjected to the electronic zoom processing is output if the second address is matched.

In the second embodiment, a target object (a human face in the second embodiment is also used) based on the image data subjected to the electronic zoom processing of enlargement or reduction stored as described above. ) Is detected, and enlargement / reduction processing is performed based on the obtained result.

The detection and processing procedure according to the second embodiment of the present invention will be described with reference to the flowchart of FIG.

First, in step S301, the RAM 3
A part of the image subjected to the electronic zoom processing stored at the second address 04 is read and input to the correlation value calculation unit 309. Note that the partial image to be read here is determined by the read address control unit 311 based on the coordinates (x, y) of the image. In the next step S302, the correlation value calculation unit 309 calculates a correlation value between the face and the template image stored in the ROM 308 in advance. Thereafter, based on the correlation value, the read address control unit 311 reads the image subjected to the electronic zoom processing while updating the read address of the RAM 304 in a predetermined procedure, and searches for a partial image whose correlation value shows a maximum value (step S30
3-S311).

It is highly probable that a part of the image when the maximum value of the correlation value is detected is an image representing a face part. Accordingly, of the images subjected to the electronic zoom processing, the partial image corresponding to the maximum value of the correlation value is sent to the discriminant analysis circuit 310, and the discriminant analysis for determining whether the face is a face is performed based on the reference data stored in the ROM 308. Is performed (step S312). If it is determined here that the face is a face, the read address control unit 311 fixes the read address, and transmits predetermined magnification data to the electronic zoom processing unit 306.
The electronic zoom processing is performed again on the output image of the camera signal processing unit 303 stored at the first address of the RAM 304 (step S313). The electronic zoom processing image at this time is an image that has been subjected to electronic zoom processing centering on the detected human face.

Next, the read address control unit 3 shown in FIG.
11 will be described in detail.

It is assumed that 0.2 is set as the threshold value in the correlation value threshold register 416. If the correlation value from the correlation value input terminal 415 is smaller than 0.2 in step S303, the correlation is low. It is determined that the face image candidate has not been detected yet, and the comparator 4
Reference numeral 17 outputs Low, which is input to one input terminal of the AND element 434. Also, the judgment result state input 432
Receives a signal 01H representing an initial state from the first A
The lower bit of the determination result state is input to the other input terminal of the ND element 434, and the first AND element 434 outputs Low. By this Low signal, the third and fourth switches 406 and 407 are connected to the 0 side (step S304). The third and fourth switches 40
The first and second switches 40 are connected to the 0-side inputs of
4 and 405, the horizontal search width data holding unit 402
(Search width Δx), vertical search width data holding unit 403
(Search width Δy) is input, and the second and third AND elements 4
37, 438 through the first and second adders 408, 4
At 09, the values are added to the value of the horizontal counter register 410 and the value of the vertical counter register 411, respectively, and set again in the counter registers 410 and 411 (step S305).

That is, when the correlation value input 415 value is smaller than 0.2, each counter register 410, 411
Will increase by each search width Δx, Δy.
Each of the search widths held by the horizontal search width data holding unit 402 and the vertical search width data holding unit 403 is 4
The type value is set in advance. This corresponds to four types of face template sizes, vertical and horizontal, and when specified by the template size control unit 401, the search width corresponding to any one of the sizes is determined by the first and second switches. 404 and 405 are selected.

In step S306, the address data generation circuit 412 determines the image position represented by the value held by each of the counter registers 410 and 411 updated as described above and the size selection state of the template size control unit 401. RAM3 according to the range of the image represented by
An address signal for designating a part of the image data subjected to the electronic zoom processing to be read from the image data 04 is generated. Therefore, RA
From M304, the image data of the designated size of the template size control unit 401 is read out while updating the address with each search width Δx, Δy, the correlation value with the template data is sequentially calculated, and the correlation value is input from the correlation value input terminal 415. Is entered.

On the other hand, if the correlation value is 0.2 or more (YES in step S303), it is determined that the face image candidate has been detected because the correlation is high, and the comparator 4
The 17 outputs become High level. Therefore, the first A
The output of the ND element 434 also becomes High level, and the third and fourth switches 406 and 407 are connected to one side (step S308). Therefore, at this point, the calculation of the correlation value performed while shifting the search widths Δx and Δy held in the horizontal search width data holding unit 402 and the vertical search width data holding unit 403 is stopped, and the correlation value is calculated in the next step S308. Start the operation of searching for the maximum value of the value. Since the operation of the local maximum value search is a method using the equations (1) to (5) described in the first embodiment with reference to FIG. 4, detailed description thereof will be omitted here.

The equations (1) to (5) are converted into a correlation value buffer 418, a subtractor 419, a coefficient generation circuit 420, a horizontal correlation differential value buffer 421, and a vertical correlation differential value buffer 42.
2, gradient direction calculation circuit 423, string function circuit 424, sine function circuit 425, first multiplier 426, second multiplier 4
27, a horizontal counter increment unit 428, a vertical counter increment unit 429, a fifth switch 430,
The calculation is executed using the sixth switch 431.
The correlation value obtained at this time and the horizontal correlation differential value buffer 4
21 and the correlation value differential values held in the vertical correlation differential value buffer 422 are input to the local maximum value determination circuit 436.

When the correlation value is not the local maximum value, the output of the local maximum value determination circuit 436 is set to a high state. Therefore, the second and third AND element outputs 437, 438
Passes the outputs of the third and fourth switches 406 and 407. When the correlation value is a predetermined value (for example, 0.4 or more) and the differential value of each correlation value is smaller than the predetermined value, it can be determined that the correlation value is the maximum value, and the maximum value determination circuit 436 is set to Low. . Thereby, the second and third AND elements 4
37 and 438 output 0, and as a result, the value of each counter register 410 and 411 is fixed.

Further, the L output from the local maximum value determination circuit 436 is
The ow signal is also input to the zoom magnification control circuit 413, and the zoom magnification control circuit 413 shifts to a zoom magnification fine adjustment operation (step S309). This is because the size of the face of the template data may be different from the size of the face of the image extracted from the RAM 304. Therefore, the size of the extracted image is gradually increased or reduced to reduce the size of the template and the size of the face image. This is done to make them match. In addition,
When the zoom magnification is not changed, the switch 321 of the correlation value calculation unit 309 shown in FIG.
The correlation value is calculated using the image blurred by the PF 320. When the zoom magnification is changed (that is, when the magnification is set to a value other than 1), the switch 321 is switched to the 0 side, and the LPF 320 The correlation value is calculated using the unblurred image. This is done to improve search speed and accuracy.

Since the correlation value becomes higher when the size of the template and the size of the face image match, step S31 is performed.
If the value is 0, the maximum correlation value detection circuit 435 detects which magnification is the maximum value of the correlation value, and feeds back the magnification at that time to the zoom magnification control circuit 413 again. The image enlarged or reduced at the magnification obtained in this way is again
M304 (step S311), and
At 312, the discriminant analysis circuit 310 performs discriminant analysis on whether or not the partial image corresponding to the detected local maximum is a face. Since the accuracy of discriminant analysis is improved as the template size and the size of the face image are more accurately matched, the size is adjusted.

When the face is determined to be a face as a result of the discrimination analysis (YES in step S312), the value of the discrimination result state input 432 becomes 11H. Therefore, at this time, the values of the counter registers 410 and 411 remain fixed to the counter values at the time when the correlation value has reached the maximum value. The higher-order bit of the determination result state signal is input to the zoom magnification control circuit 413, and High is input here for the first time. Then, the zoom magnification control circuit 413 outputs the original electronic zoom magnification data, and the electronic zoom operation centering on the face portion can be performed (step S31).
3). On the other hand, as a result of the discrimination analysis, when it is determined that the face is not a face (NO in step S312), the value of the discrimination result state signal is 00H. Therefore, the first AND element 434
Becomes zero, and the third and fourth switches 406 and 4
07 is forcibly set to the 0 side connection (step S30).
4). At this point, the search widths Δx and Δy held in the horizontal search width data holding unit 402 and the vertical search width data holding unit 403 are added to the values of the counter registers 410 and 411, respectively. The search is resumed.

In the first and second embodiments, a mechanical configuration has been described. However, a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded. , Supply to the device, computer of the device (or CPU or MPU)
Can also be achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. When the computer executes the readout program codes, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instructions of the program codes. It goes without saying that a case where some or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

Further, after the program code read from the storage medium is written into the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, the program code is read based on the instruction of the program code. , The CPU provided in the function expansion card or the function expansion unit performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts shown in FIGS. 3 and 4 or FIG. 7 described above.

[0072]

As described above, according to the present invention,
A specific image pattern of a target object such as a human face in a captured image is detected by a template matching method, and automatic focus control and automatic exposure control are performed based on a portion of the captured image corresponding to the detected specific image. Since the automatic white balance control and the electronic zoom control can be performed, it is possible to effectively perform the photographing under the above-described respective controls even when the subject is not located at the center of the screen.

Further, according to the present invention, even if a strong correlation is detected with respect to a background similar to the above-mentioned specific pattern by the template matching method for a natural image, the specific pattern is truly obtained by performing a discriminant analysis. Only in this case can final pattern detection be performed.

Further, according to the present invention, by proposing an efficient search method of the template matching method, it is possible to realize a search with a calculation amount operable even at a video frame rate.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an imaging device according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration of a data read control circuit shown in FIG.

FIG. 3 is a flowchart illustrating an operation of the imaging apparatus according to the first embodiment of the present invention.

FIG. 4 is a flowchart illustrating a procedure of a local maximum value detection process according to the first embodiment of the present invention.

FIG. 5 is a block diagram illustrating a configuration of an imaging device according to a second embodiment of the present invention.

FIG. 6 is a block diagram showing a detailed configuration of a read address control unit shown in FIG. 5;

FIG. 7 is a flowchart illustrating an operation of the imaging apparatus according to the second embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 101 imaging optical system 102 CCD image pickup device 103 A / D converter 104 camera signal processing unit 105 video signal output terminal 106 random access memory 107 memory interface circuit 108 read-only memory storing template image 109 correlation value calculation unit 110 discriminant analysis circuit Reference Signs List 111 Data reading control means 112 AF, AE, AWB evaluation value detection circuit 113 AF, AE, AWB control circuit 114 Template size control circuit 301 CCD image pickup device 302 A / D converter 303 Camera signal processing unit 304 Random access memory 305 Memory interface circuit 306 Electronic zoom processing unit 307 Video output processing unit 308 Read-only memory 309 Correlation value calculation unit 310 Discrimination analysis circuit 311 Read address control unit 312 Oh signal output terminal

Claims (21)

[Claims] 1. An image pickup apparatus for converting an optical image obtained through an image forming optical system into an electric image signal by a photoelectric conversion element and storing the electric image signal, wherein: a storage unit for storing a specific image pattern; A partial image of the obtained image signal, a correlation value calculation unit for calculating a correlation value of the specific image pattern, a maximum value detection unit for detecting a maximum value of the correlation value calculated by the correlation value calculation unit, the maximum value An imaging apparatus comprising: a control unit that performs predetermined control based on a partial image corresponding to a local maximum value detected by a detection unit. 2. The method according to claim 1, wherein the local maximum value detecting unit compares the correlation value calculated by the correlation value calculating unit with a predetermined value, and determines that the correlation value is lower than the predetermined value. First shifting means for shifting the position of the partial image of the image signal by a predetermined amount; and a direction in which the correlation value changes most when the comparing means determines that the correlation value is equal to or more than a predetermined value. A second shift means for shifting the position of the partial image of the image signal in the calculated direction, and a correlation value between the partial image at the position shifted by the second shift means and the specific image pattern. The imaging apparatus according to claim 1, further comprising a maximum value detection unit that detects a maximum value. 3. An electronic zoom unit for enlarging / reducing the image signal, wherein the local maximum value detecting unit specifies the partial image corresponding to the correlation value of the maximum value detected by the maximum value detecting unit and the identification. 3. The imaging apparatus according to claim 2, further comprising a unit that sets a magnification of the electronic zoom unit so that a correlation value with an image pattern is maximized. 4. A determining unit that determines whether a partial image corresponding to a local maximum value detected by the local maximum value detecting unit is a target object indicated by a specific image pattern, and the determining unit includes: Means for deactivating the control means and activating the first shift means to shift the position of the partial image of the image signal by a predetermined amount when it is determined that the partial image is not the target object The imaging device according to claim 2, further comprising: 5. The imaging device according to claim 1, wherein the predetermined control includes at least one of an automatic focus control, an automatic exposure control, and an automatic white balance control. 6. The imaging apparatus according to claim 1, wherein the predetermined control includes at least one of enlargement processing and reduction processing. 7. The storage means stores a plurality of similar image patterns having different sizes as a specific image pattern,
7. The imaging apparatus according to claim 1, further comprising a selection unit that selects one of the plurality of image patterns stored in the storage unit. 8. An electronic zoom unit for enlarging / reducing the image signal, wherein the correlation value calculating unit includes: a partial image of the image signal enlarged or reduced by the electronic zoom unit; The imaging apparatus according to claim 1, wherein the correlation value is calculated. 9. The image signal used for detecting a position where the correlation value has a maximum value in the local maximum value detecting means uses an image blurred by a low-pass filter or the like, and a magnification of the electronic zoom means. The image pickup apparatus according to claim 3, wherein the image signal used for obtaining the maximum value of the correlation value when the image is changed is an unblurred image. 10. The imaging device according to claim 1, wherein the imaging device is a video camera. 11. A method for controlling an imaging apparatus, wherein an optical image obtained through an imaging optical system is converted into an electric image signal by a photoelectric conversion element and stored, and a specific image pattern is stored in advance. A partial image of the obtained image signal, a correlation value calculating step of calculating a correlation value of the specific image pattern, a local maximum value detecting step of detecting a local maximum value of the correlation value calculated in the correlation value calculating step, the local maximum A control step of performing predetermined control based on a partial image corresponding to a local maximum value detected in the value detection step. 12. The local maximum value detecting step includes a comparing step of comparing the correlation value calculated in the correlation value calculating step with a predetermined value, and a case where the correlation value is determined to be lower than the predetermined value in the comparing step. A first shift step of shifting the position of the partial image of the image signal by a predetermined amount; and a direction in which the correlation value changes most when the correlation value is determined to be equal to or more than the predetermined value in the comparing step. A second shift step of shifting the position of the partial image of the image signal in the calculated direction, and a correlation value between the partial image at the position shifted by the second shift step and the specific image pattern. 12. A maximum value detecting step of detecting a maximum value.
The control method of an imaging device according to claim 1. 13. An electronic zoom step of enlarging / reducing the image signal, wherein the local maximum value detecting step comprises: identifying the partial image corresponding to the correlation value of the maximum value detected in the maximum value detecting step; 13. The method according to claim 12, further comprising a step of setting a magnification in the electronic zoom step such that a correlation value with an image pattern is maximized. 14. A judging step of judging whether or not a partial image corresponding to the maximal value detected in the maximal value detecting step is a target object indicated by a specific image pattern; When it is determined that the partial image is not the target object, the control step is deactivated and the first shift step is activated to shift the position of the partial image of the image signal by a predetermined amount. The method according to claim 12 or 13, further comprising a step. 15. The predetermined control is at least one of automatic focus control, automatic exposure control, and automatic white balance control.
The method according to claim 11, further comprising: 16. The method according to claim 11, wherein the predetermined control includes at least one of an enlargement process and a reduction process. 17. The image pickup apparatus stores a plurality of similar image patterns having different sizes as a specific image pattern, and further comprises a selecting step of selecting one of the plurality of image patterns. A method for controlling an imaging device according to claim 11. 18. An electronic zoom step of enlarging / reducing the image signal, wherein in the correlation value calculating step, a partial image of the image signal enlarged or reduced in the electronic zoom step and the specific image pattern 18. The method according to claim 11, wherein the correlation value is calculated. 19. The image signal used for detecting a position where the correlation value has a maximum value in the local maximum value detecting step uses an image blurred by a low-pass filter or the like, and a magnification of the electronic zoom means. 13. The image signal used for obtaining the maximum value of the correlation value when the image is changed is an unblurred image.
9. The control method of the imaging device according to 8. 20. The method according to claim 11, wherein the control method of the imaging device is a video camera. 21. A storage medium for holding a program code for realizing the control method for an imaging device according to claim 11.