JPH03154576A - Picture recognition device - Google Patents

Abstract

PURPOSE:To prevent generation of a malfunction in picture processing due to a background other than a major object or the like by recognizing a picture pattern of a focused part in a photographing pattern in two dimension and in real time. CONSTITUTION:A detection means 10 detecting a signal representing a focus to an object from a picture signal outputted from an image pickup means 2 and a means 12 recognizing the two dimensional distribution state of the focus detected by the detection means 10 on the pickup face are provided and the shape and position of the two dimensional pattern of the focused object are recognized in a real time and a focus detection area or the like is set adaptively. Thus, the object is continuously focused without being affected by the object other than the target.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an image recognition device suitable for use in a camera that converts an optical image into an electrical signal using a two-dimensional image sensor or the like.

[Conventional technology]

In video cameras, there is a method that automatically controls the camera to be in focus by detecting the definition of the shooting screen using the high frequency components of the video signal and controlling the focusing lens position so that the high frequency components are maximized. Are known. Specifically, the video signal changes rapidly at the edges of the subject image, and the high frequency components of the video signal increase. and,
The higher the amplitude of this high frequency component, the closer the subject image is to being in focus.

FIG. 11 shows a schematic configuration of a conventional automatic focusing device.

In FIG. 11, an optical image of a subject is formed by a lens 1 on the imaging surface of an image sensor 2, and the image sensor 2 converts it into an electrical signal. A preamplifier 3 amplifies the output video signal of the image sensor 2, and a process circuit 4 performs predetermined signal processing and outputs it as a standard TV signal.

The bandpass filter 5 extracts high frequency components from the output of the preamplifier 3, and the gate circuit 6 extracts high frequency components from the signal for one screen (i.e., l field or one frame) for the area where focus detection is performed (so-called ranging area). Selects and passes only signals. The detection circuit 7 detects the output of the gate circuit 6 and forms a signal indicating the maximum amplitude value, ie, the peak value, of the high frequency component on the screen. The output signal of the detection circuit 7 represents the degree of focus of the lens 1, and the larger the output signal, the closer to the in-focus state it becomes. The motor drive circuit 8 drives the motor 9 according to the output value of the detection circuit 7 for each photographic screen, and automatically controls the lens 1 to be in focus.

[Problem that the invention seeks to solve]

In this conventional automatic focusing device, there are examples in which the passage area of the gate circuit 6 is set to a wide fixed area within the screen, and examples in which it is set to a slightly narrow fixed area in the center of the screen. . In the former case, the peak value of high-frequency components within the screen is obtained and the goldfish signal is formed in an extremely narrow spot area.
Another subject may enter the focus detection area, or the camera may move and the focus detection area itself may move and another subject may enter the focus detection area. Sometimes I end up focusing. In actual photography, the subject that the photographer wants to photograph is usually the same unless it is intentionally changed, so such an operation is undesirable.

On the other hand, when the gate circuit 6 passes only through a narrower area, there is a lower possibility that other subjects will enter the focus detection area, but there is a possibility that the target subject will move out of the focus detection area. As a result, the focusing operation tends to have to be restarted frequently. From the photographer's point of view, this is recognized as a defect in the focusing operation, and must be avoided as much as possible.

(Means for Solving the Problems) The present invention was made in order to solve the above-mentioned problems, and provides an image recognition device that can continuously focus on a target object without being affected by objects other than the target object. The purpose is to provide.

In order to achieve such an objective, the present application includes an imaging means for converting a subject image formed on an imaging surface into an image signal and outputting the image signal, and a method for focusing the subject image from the image signal output from the imaging means. detection means for detecting a signal representing the degree;
and means for recognizing a two-dimensional distribution state of the degree of focus detected by the detection means on the imaging plane.

(Function) As a result, the shape and position of the two-dimensional pattern of the object that is in focus on the imaging screen can be recognized in real time, and the focus detection area and the like can be adaptively set.

(Example) The image recognition device according to the present invention will be described in detail below with reference to each figure.

FIG. 1 is a block diagram showing the case where the present invention is applied to a video camera. Components that are the same as those of the conventional example shown in FIG. 11 are given the same reference numerals, and their explanations will be omitted.

In FIG. 1, 10 is a blur width detection circuit that detects the blur width in the video signal output from the preamplifier 3, that is, the width corresponding to the blur amount of the edge portion of the subject image; It recognizes the distribution state of the blur width of the edge portion on the imaging screen according to the degree of focus of the subject image, and outputs to the gate circuit 6 an area control signal for setting a passband, that is, a focus detection area on the imaging screen. This is a blur width distribution recognition circuit, and the output of the blur width detection circuit 10 is
It is composed of a binarization circuit 12 that converts into values, and a smoothing circuit 13 that smoothes the binarized output of the binarization circuit 12.

With the above configuration, the subject image formed on the imaging surface by the lens l is converted into a video signal by the image sensor 2,
The preamplifier 3 amplifies the signal to a predetermined level, and the process circuit 4 performs processing such as adding a synchronization signal, gamma correction, and adding a color burst signal, and then outputs it as a standardized television signal.

On the other hand, the video signal (synchronization signal,
(without gamma correction, color burst signal addition, etc.) is input to a band pass filter 5, where high frequency components that increase or decrease according to a predetermined degree of focus are extracted, and then displayed on the imaging screen by a gate circuit 6. Only the signal corresponding to the set focus detection area is passed to the detection circuit, and the position of the lens I is adjusted by mountain climbing control by driving the motor 9 in a direction in which the detection level of the high frequency component increases.

Further, the video output of the preamplifier 3 is supplied to a blur width detection circuit 10 to detect the blur width of the edge portion of the subject image, and a signal corresponding to this is output serially. This blur width detection output signal is supplied to the blur width distribution recognition circuit 12, the edge width signal is binarized by the binarization circuit 11, smoothed by the smoothing circuit 13, and sent to the gate circuit 6 to It is supplied as an area control signal for setting a passing area.

That is, according to the apparatus of the present invention, it is possible to adjust the focus of the photographing lens and set the focus detection area in real time while photographing a subject image. Here, a specific configuration of the blur width detection circuit IO will be explained using an example. As mentioned above, the blur width detection circuit outputs a signal corresponding to the width of the edge portion of the subject image, which changes depending on the degree of focus.The detection output of this edge width is based on the contrast and brightness of the subject, It represents the sharpness of the subject image, which does not depend on the pattern of the image. A specific example of such a system is also shown in, for example, Japanese Patent Laid-Open No. 103616/1983.

This method accurately evaluates only the sharpness of the object image, regardless of the contrast of the object, by detecting the width of the edge portion of the object image. Here, the configuration of the edge width detection circuit will be explained using FIGS. 2 and 3.

FIG. 2(a) shows an imaging screen in which the subject 100 is imaged. On this screen, if we illustrate the luminance change of the video signal on the straight line l, for example, Fig. 2(b)
become that way. The vertical axis shows the brightness level, and the horizontal axis shows the position on the screen. Assuming that the object 100 is in focus, the brightness level in the video signal of the object 100 is high, and the brightness level of the background is low. The same applies to high frequency components in the luminance signal. Now, attention is paid to the edge portion of the subject image, and the width of the edge portion is ΔX, the brightness difference is ΔX1, and the edge width is Δ■.

The width ΔX of this edge portion becomes smaller as it approaches the in-focus point, increases as it becomes out of focus, and takes a minimum value at the in-focus point.

This ΔX is determined by the diameter of the circle of confusion of the optical system, the resolution of the image sensor, and the bandwidth of the image signal processing system, but it is unrelated to the focusing and defocusing of the latter two optical systems. , the diameter of the former circle of confusion changes depending on the in-focus and out-of-focus states.

However, it is not affected by the subject's situation or brightness. Therefore, by detecting this edge width ΔX and comparing it with the diameter of the circle of least confusion, it is possible to accurately determine whether the object is in focus or not.

FIG. 3 is a block diagram showing the internal configuration of the blur (edge) width detection circuit 10.

In the figure, 101 is a differentiation circuit that differentiates the video signal output from the preamplifier 3 to obtain d 1 / d In the circuit to find Δ
I can be obtained by integrating d 1 /d x in the X direction in a minute interval.

104 is a delay circuit that delays ldl/dxl by the delay time and adjusts the calculation timing based on the integration for calculating Δ■; 105 is a delay circuit that uses the output 1dl/dxl of the delay circuit 104 to This is a division circuit that calculates the edge width ΔX in the form of a reciprocal number by dividing the output ΔI by the output ΔI and calculating (di/dx)/ΔI=1/ΔX.

Since the blur width detection circuit 10 outputs the edge width ΔX in the form of a reciprocal number, the larger this value is, the closer the edge width is to the in-focus point.

Information based on the edge width itself is not affected by the size, brightness, or contrast of the subject in principle, and the blur width of the edge portion can be detected in real time from the video signal.

When the video signal Sa is serially input from the preamplifier 3 to the blur width detection circuit 1o, a pulsed blur width detection signal sb corresponding to the edge portion of the subject image is output from the video signal, and this blur width detection The signal is binarized circuit 1
1, the value is compared with a preset value and binarized. This setting value is set to several times the minimum blur width (minimum circle of confusion diameter) determined by the imaging performance of the lens, and the blur width of the edge portion obtained by the blur width detection signal is less than this setting value. If so, it is determined that the portion is in focus or close to the in-focus point and is set to “1”; if it is equal to or greater than the set value, it is determined that it is an out-of-focus portion and is set to “0”.

This binary signal is delayed by the smoothing circuit 13.

By performing calculations such as integration, removing noise components, and performing expansion (thickening) processing, the approximate area including the edge portion of the in-focus subject image and the area in its vicinity is shown in the figure. It can be obtained by a pulse waveform as shown by Sc.

The output waveform of the edge partial region of the smoothing circuit 13 is supplied to the gate circuit 6 and acts as a region control signal for setting the region on the imaging screen through which the high frequency component output of the bandpass filter 5 is passed.

That is, if the area through which this area control signal passes on the imaging screen is illustrated, the area includes the edge portion of the object in focus and the area around it, as shown by diagonal lines in FIG. FIG. 5A shows an image of a subject being photographed, and it is assumed that the main subject, such as a person present in the center, is currently in focus, and the background is out of focus. In this case, as is clear from the above explanation, the edge portion of the object that is in focus has a small blur width (the output of the blur width detection circuit 10 is output from the binarization circuit 11 in the blur width distribution recognition circuit 12). It is determined to be "1", is thickened through the smoothing circuit 13, and is output to the gate circuit 6 as a region control signal pulse (C) in FIG.

The gate circuit 6 is opened by this region control signal pulse and passes the output of the bandpass filter 5 to the detection circuit 7. Therefore, in the video signal, only high frequency components corresponding to the edge portions of the in-focus subject image are detected and used as signals for focus control.

On the other hand, since the out-of-focus portion has a large blur width, the output waveform of the blur width detection circuit is determined to be "0" by the binarization circuit 11 and is not supplied to the smoothing circuit 13. Therefore, for the out-of-focus portion, a region control signal pulse for opening the gate is not supplied to the gate circuit 6, and a passing region, that is, a focus detection region is not set.

From the above, in the case of a shooting screen where the main subject image such as a person is in focus and the background is in focus as shown in Figure 4(a), the shaded area in Figure 4(b) is shown. The gate circuit 6 is opened only in the edge portion of the in-focus subject image in the range indicated by the thick line by the smoothing circuit 13, and a focus detection area is set. Since automatic focus adjustment is performed according to the increase or decrease of the high frequency component corresponding to the focus detection area, it is possible to reliably focus on the main subject to be photographed without being affected by the background.

As a result, even if the main object to be photographed moves or camera shake occurs, the main object can be followed (tracked) and kept in focus as long as it is within the imaging screen.

Note that when recording images taken with a video camera using a video tape recorder (not shown), the photographer usually starts recording after confirming that the camera is focused on the subject he or she intends to photograph (the main subject). Even if the main subject moves thereafter, the focus detection area is automatically set to that part of the subject, so the subject can continue to be in focus.

From the above, by linking the setting of the focus detection area recognized by the device of the present invention with the recording button and tracking the image pattern at the start of recording, it is possible to prevent near-far conflicts.

Also, when the depth of field of the lens is deep, the difference in the width of the blur between the main subject and the background is small, reducing the discrimination accuracy. do not become.

Furthermore, if there is no in-focus subject within the image capture screen, the function of the present device may be turned off and a predetermined, for example, rectangular focus detection area may be set at the center of the image capture screen.

The block diagram in Figure 1 has been omitted because it shows the principle of the present invention, but it realizes a configuration that links the recording button and image recognition, and a configuration that sets the focus detection area when there is no in-focus subject. The block diagram in FIG. 5 shows a configuration in which these control circuits 14 are added for this purpose. Portions drawn the same as those in FIG. 1 are designated by the same reference numerals, and a description thereof will be omitted.

In the figure, numeral 14 records the video signal output from the process circuit 4. video tape recorder (VTR),
15 is a recording start button that instructs the VTR to start recording;
Reference numeral 16 denotes a control circuit that controls the passage area of the gate circuit 6 in response to operations of the blur width distribution recognition circuit 12, the recording start button 15, and the like.

FIG. 6 is a flowchart showing the focus detection area setting operation. When the control flow starts, first, in step S1, the video signal output from the preamplifier 3 (see Fig. 1) is
From a)), the blur width detection circuit 10 outputs the blur width signal (FIG. 1(b)), and in step S2, the blur width distribution recognition circuit 12 obtains the area control signal shown in FIG. 1(c), It is output to the control circuit 16.

The control circuit 16 detects the area control signal within the image capture screen in step S3, determines whether or not there is an in-focus area, and if the in-focus area does not exist within the screen, the process proceeds to step S8.
After proceeding to , a predetermined shape, for example, a rectangular focus detection area is set in the center of the screen, the process proceeds to step S9, where it is checked whether the record button 15 is pressed, and if it is ON, the process proceeds to step S6, where recording is started. , and continues recording until the recording button is turned off in step S7.

On the other hand, if the area control signal (FIG. 1(C)) is output in step S3 and there is an in-focus area, the process proceeds to step S4, where it is determined whether or not the record button 15 is turned on. If 15 is not ON, step S
Returning to step 1, if the recording button has been turned on, the process advances to step S5, and recording is started in step S6 using the focus detection area set based on the area control signal at the time the recording button was turned on as an initial area. .

If recording is started using the focus detection area at the time of operation of the record button as the initial area (9), it is possible to keep the focus on the initial object to be photographed throughout the recording.

Furthermore, when there is no in-focus area, by resetting the focus detection area to the center of the screen, there is no possibility that the focus adjustment operation will malfunction due to an unnecessary object.

Next, a case will be described in which the image recognition device according to the present invention is used to set a motion detection area of an image and is applied to, for example, an anti-shake device for a video camera or the like.

In hand-held video cameras, especially those equipped with long focal length lenses, vibrations caused by the photographer's walking, breathing, and vehicles cause the screen to shake, resulting in unsightly images. Therefore, an anti-shake camera has been proposed which detects the movement of an image from an image signal and corrects it (for example, Japanese Patent Laid-Open No. 61-269572).

When detecting motion in an image using such image signals, it is necessary to accurately set the motion detection area, which part of the image should be detected and which part should be ignored. There is.

FIG. 7 is a block diagram showing an example of the configuration of an anti-shake camera in which a motion detection area is set using the image recognition device of the present invention.

In this figure, the same reference numerals are used for the same components as those in FIG. 1, and the explanation thereof will be omitted.

In the figure, reference numeral 16 denotes a gate circuit that sets an image motion detection area for image stabilization on the imaging screen and passes only the edge signal output of the blur width detection circuit 10 corresponding to the detection area; 17.18 19 is a horizontal and vertical edge counting circuit that counts and stores edge signals corresponding to the motion detection area extracted by the gate circuit 16 in the horizontal scanning direction and vertical scanning direction of the screen as feature points; 20 compares the count outputs of the edge count circuits in the horizontal and vertical directions between screens at different times, and calculates the horizontal direction for the image pattern.

These are horizontal and vertical correlation circuits that create a histogram by correlating the distribution of the number of edges in the vertical direction, and output signals corresponding to the direction and amount of image movement in the horizontal and vertical directions, respectively.

21 is based on external information such as focal length information of the photographing lens optical system, object distance information, focus state information, etc., on signals corresponding to the moving direction and moving amount of the image output from the horizontal and vertical correlation circuits 19 and 20. This is a correction circuit that applies correction based on the

This is because it is necessary to change the amount of image pre-correction, the correction speed, etc. in accordance with these external information.

22 is a variable apex angle prism that corrects blurring by changing the angle of the optical axis of the photographing lens optical system; 23 is an actuator that drives and controls this variable apex prism. The variable apex angle prism is, for example, one in which transparent silicon resin is sealed between two parallel glass plates, and the direction of the optical axis is changed by varying the angle of the two glass plates with an actuator 23. It is.

Reference numeral 24 denotes a drive circuit that drives the actuator 23 in a direction to correct image blur based on the corrected control signal output from the correction circuit 21.

Further, 25 is a gate circuit 1 for setting an image motion detection area based on the output of the smoothing circuit 13 of the blur width distribution recognition circuit 12.
6 is an EX-OR (exclusive OR) circuit, and 26 is an area setting switch that controls the EX-OR circuit 25 to select and operate the image motion detection area by the gate circuit 16.

As explained in the first embodiment, the output of the blur width distribution recognition circuit 12 has rough edge information of the object in focus. At this time, it is desirable to set the filter size of the smoothing circuit larger than that in the first embodiment to obtain a high noise removal effect.

When the area setting switch 26 is OFF, EX-OR
The circuit 25 gate B becomes “1”, and the EX-OR circuit 2
6 operates as an inverter, and a blur width distribution recognition circuit 12
The area control signal output from the smoothing circuit 13 to the gate circuit 6 is inverted and supplied to the gate circuit 16. Therefore, the image motion detection area set by the gate circuit 16 is an area outside the focus detection area set by the gate circuit 6.

Assuming that the gate circuit 6 selects the hatched area in FIG. 8(b) as the focus detection area on the screen shown in FIG. 8(a), the movement of the image The detection area is the area shown by diagonal lines in FIG. 8(C), which corresponds to the outside thereof. That is, camera shake is detected based on image movement information in an area that is the background to the main subject being focused on, and the variable apex angle prism 22 is driven to perform shake correction. That is, during shooting, even if the subject moves unless the camera is moved, the background remains stationary, so if the movement of the background is detected, camera movement and blur can be detected. Therefore, even if the position or shape of the main subject changes, the image motion detection area for image stabilization will always be outside the focus detection area that captures the subject, so subject movement may be mistakenly detected as camera shake. It is possible to accurately detect and correct camera shake without any problems.

Furthermore, even if the position of the subject changes within the screen, the focus detection area is automatically tracked and set according to the outline of the subject, as described above, so accurate movement detection can always be performed. becomes.

Further, when the area setting switch 26 is ON, the gate B becomes "O", and the EX-OR circuit 25 becomes through. That is, since the area control signal output from the smoothing circuit 13 of the blur width distribution recognition circuit 12 and supplied to the gate circuit 6 is also supplied to the gate circuit 16 as is, the gate circuit 1
The image motion detection area set by the gate circuit 6 is the same as the focus detection area set by the gate circuit 6 as shown in FIG. 8(b).

In this case, since the movement of the main subject with respect to the camera screen is detected and corrected, it is suitable for photographing a moving subject by moving the camera to follow it. In other words, when tracking a moving subject by moving the camera, the main subject always tries to be captured within the frame, so the change in the position of the main subject within the captured frame is small (the background is flowing, therefore In order to correct camera shake, it is necessary to perform shake detection and pre-correction on the image of the main subject whose position relative to the camera is small. .

In this way, when photographing a subject with the camera stationary, by turning off the switch 26, camera shake is detected from the movement of the background image and corrected, and the camera is moved. When shooting while following a subject, by turning on the switch 26, camera shake can be detected from the movement of the subject and the relative shake between the subject and camera can be corrected, and the camera can be switched as appropriate depending on the shooting mode. It can be used as

Further, a case will be considered in which the image recognition device according to the present invention is used for motion detection of a video camera.

For example, the image motion detection means shown in FIG. 7 detects motion based on the correlation of a histogram of the number of feature points of an image.

When edges of image patterns are used as feature points in this way, edges are generally extracted based on the density level of a multi-level density gradation image, but with this method, when the intensity of illumination light changes, The position changes, causing errors in motion detection.

Therefore, according to the fourth embodiment of the present invention shown in FIG.
Instead of using the edge information from the blur width detection circuit IO as is, the binarization circuit 11 in the blur width distribution recognition circuit 12 compares the edge information with a predetermined threshold value, that is, the edge width of the image is smaller than the threshold value. Only in-focus areas with small blur width are recognized as feature points. That is, the configuration is the same as in FIG. 7 except that the output of the binarization circuit 11 is supplied to the gate circuit 16, and the explanation thereof will be omitted.

Figures 9(a) and 9(b) show this situation. Figure 9(a) shows the image captured on the screen, and Figure 9(a) shows the image captured on the screen.
b) shows only a portion where the blur width is smaller than that of the image pickup screen shown in FIG. The image signal processing for setting this area has the same configuration as the above-mentioned embodiment, but the blur width distribution recognition circuit 12
It is desirable that the filter size of the smoothing circuit 13 is small in order to remove noise.

Reference numeral 51 indicates a histogram in the horizontal (X) direction calculated from the edge information obtained through the binarization circuit 11, and 52 indicates a histogram in the vertical (Y) direction.

Then, as in the above-described embodiment, it is possible to detect changes in the image pattern due to changes in time from these histograms, and to detect the movement of the image.

Incidentally, FIG. 9 shows an image pattern when the area selection switch 26 in FIG. 8 is turned on and motion detection regarding the main subject is performed. That is, it is possible to detect the movement of the subject, drive the variable apex angle prism so as to offset the movement, and track the subject so that it is always located at the same position within the screen.

In this way, in the present invention, edges are recognized based on the edge width rather than edge strength, so the edge position does not change depending on the intensity of illumination, that is, brightness, and accurate and stable image motion detection is possible. It can be performed.

In this embodiment, the present invention is applied to a subject tracking device for an anti-shake camera, but the present invention is not limited to this.
It can be widely used as an image motion detection device.

[Effects of the Invention] As described above, according to the image recognition device of the present invention, the image pattern of the in-focus portion within the imaging screen can be recognized two-dimensionally and in real time. It is possible to realize an extremely accurate and reliable image recognition device that does not cause malfunctions in image processing due to backgrounds other than the main subject, and can be used as an image detection area setting means for automatic focus adjustment devices, automatic image stabilization devices, etc. can.

In addition, the present invention can realize image recognition entirely by processing image signals, can be configured with electrical components, and by converting it into an LSI, it can be installed in a video camera, etc. as a small and lightweight unit, and can be used for hand-held shooting. This is extremely effective for small video cameras and the like.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing a first embodiment in which the image recognition device of the present invention is applied to an automatic focus detection device such as a video camera; FIG. 2 is a diagram for explaining the operation of a blur width detection circuit; 3 is a block diagram showing the configuration of the blur width detection circuit, FIGS. 4(a) and 4(b) are diagrams for explaining the image recognition operation according to the first embodiment of the present invention, and FIG. 5 6 is a block diagram showing a second embodiment in which the image recognition device of the present invention is applied to a video camera, FIG. 6 is a flowchart for explaining the operation of the embodiment of FIG. 5, and FIG. 7 is an image recognition device according to the present invention. A block diagram showing a third embodiment in which the recognition device is applied to an anti-shake camera, FIG. 8(a),
FIG. 8(b) and FIG. 8(c) are diagrams for explaining the operation of the third embodiment shown in FIG. 7, and FIG. 9 is a diagram for explaining the operation of the third embodiment of the present invention. FIG. 1O is a diagram for explaining the operation of the embodiment of FIG. 9. FIG. 11 is a block diagram showing an example of an automatic focus adjustment device in a conventional video camera. (α〕 (b) Shibe) Ray figure (of) Shooting screen) (Doretoka Pa1 go) Tesu 6) A4 Figure (b) 8 screen hand figure b ri 8 Figure 2/θ Figure, - m = (two times)

Claims (1)

[Claims] an imaging means for converting a subject image formed on an imaging surface into an image signal and outputting the image signal; a detection means for detecting a signal representing a degree of focus on the subject image from among the image signals output from the imaging means; An image recognition device comprising: means for recognizing a two-dimensional distribution state of the degree of focus detected by the detection means on the imaging surface.