JP3192483B2 - Optical equipment - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention relates to an image processing method based on an image signal.
The present invention relates to an optical device for extracting a region where an object exists .

[0002]

2. Description of the Related Art Conventionally, an area has been set in a photographing screen when performing various photographing controls. For example,
In autofocus, a focus detection area for detecting focus information is set, and in image blur correction due to camera shake of a photographer, a shake detection area for detecting image movement is set. Each photographing control is performed based on the image pickup signal in the set area.

Further, there has been proposed a camera capable of automatically tracking a previous focus detection area on a subject based on focus information of the subject or the like.

[0004]

However, in the above-described conventional example, the area set at the time of shooting control is configured to be set in the screen in advance, and the photographer sets the area at an arbitrary position as necessary. I couldn't do that, and the shooting range was limited. For this reason, in the case of setting a subject area (range) to be extracted based on information of the subject such as focusing information, the area is set depending on whether or not the subject is in focus. select an object different from the object intended by the, i.e. there is a problem that becomes in focus on the intended object.

[0005] Various types of editing apparatuses have been proposed in which an area is set after photographing and images are synthesized as an editing operation. However, an area can be set at an intended position on a screen as described above. In some cases, it is necessary to prepare an editing device separately, it is necessary to spend extra time working, and only for the image after shooting.
Or region setting information has a problem such can not be used.

In view of the above, an object of the present invention is to extract
If there is more than one area, select the intended area.
It is to provide an optical device that can be used .

[0007]

Means for Solving the Problems To achieve the above object,
According to the present invention, an image signal is output.
Image sensor, and presence of an object based on the image signal
Extraction means for extracting a region to be extracted, and extraction by the extraction means
Area selection to select at least one of the selected areas
Means. Claim 2
An image sensor for outputting an image signal
And extracting the area where the object exists based on the image signal.
Extraction means for outputting, gaze detection means for detecting gaze,
The extraction means based on a gaze detection result of the gaze detection means;
Select at least one of the regions extracted by
Region selecting means. Ma
The present invention described in claim 3 provides the invention according to claim 1 or 2.
In addition to the above invention, the extraction means corresponds to the size of the object.
It is characterized in that a region to be extracted is extracted.

[0008]

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the illustrated embodiments.

FIG. 1 is a block diagram of a photographing apparatus according to a first embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a photographing lens, 2 denotes an image pickup device such as a CCD, and 3 denotes an image pickup signal from the image pickup device 2. A subject extraction circuit 4 extracts (sets) a region (range) of the subject in the screen. A subject frame generation circuit 4 generates a subject frame based on the region information extracted by the subject extraction circuit 3. A finder control circuit that combines the subject frame generated by the frame generation circuit 3 with a video signal and displays the synthesized image on a finder; 6, a display device such as an electronic view finder; 7, a finder lens for observing the finder; Is a line-of-sight detection optical system that detects the line of sight of the photographer from the movement of the eyeball of the photographer, 9 is a detector that detects an image of the eyeball, and 10 is an optical path of the finder system and an optical path of the line-of-sight detection system. An optical path separating mirror for separation, 11 is a line-of-sight extraction circuit for extracting the position of the line of sight, 12 is a main object determined based on the area information from the object extraction circuit 3 and the line of sight information from the line of sight extraction circuit 11, and control of the object information A video signal conversion circuit 13 for converting an image signal from the image sensor 2 into a video signal conforming to a television standard; a photographing lens driving circuit 14 for controlling the driving of the photographing lens 1; An image sensor driving circuit 16 controls the exposure time, shutter speed, and the like of the image sensor 1, and a synchronization signal generating circuit 16 generates a signal for synchronizing the operation of each circuit.

The operation of the object extracting circuit 3 will be described with reference to FIG.

The subject extracting circuit 3 detects a blur width corresponding to the blur amount of the edge portion of the subject image from the image pickup device 2 (see FIG. 2A), and blurs the edge corresponding to the detected degree of focus. The distribution of the width on the imaging screen is recognized (see FIG. 2 (b)), and the area of the main subject is set from the in-focus distribution (see FIG. 2 (c)).

Next, the basic principle of a gaze detection method using a gaze detection device including the above-described gaze detection optical system 8, a detector 9 such as an image sensor, and a gaze extraction circuit 11 will be described with reference to FIG.

In FIG. 3, reference numeral 104 denotes a light source such as a light emitting diode which emits infrared light insensitive to an observer;
It is arranged on the focal plane of the light projecting lens 106.

The infrared light emitted from the light source 104 is converted into parallel light by the light projecting lens 106, reflected by the half mirror 105, and illuminates the cornea 101 of the eyeball. At this time, the cornea 1
Corneal reflection image d based on a part of the infrared light reflected on the surface of No. 01
Is transmitted through the half mirror 105, condensed by the light receiving lens 107 corresponding to the visual axis detection optical system 8 in FIG. 1, and is reflected at the position d ′ on the image sensor 109 also corresponding to the detector 9 in FIG. Is re-imaged.

The light beams from the ends a and b of the iris 103 are guided to the image sensor 109 via the half mirror 105 and the light receiving lens 107, and the positions a 'and b' thereof.
Then, the images of the ends a and b are formed. When the rotation angle θ of the optical axis of the eyeball with respect to the optical axis of the light receiving lens 107 is small, and the Z coordinates of the ends a and b of the iris 103 are Za and Zb,
The coordinate Zc of the center position c of the iris 103 is expressed as Zc ≒ (Za + Zb) / 2.

If the Z coordinate of the corneal reflection image generation position d is Zd, and the distance between the center of curvature O of the cornea 101 and the center C of the iris 103 is a distance OC, the rotation angle θ of the optical axis of the eyeball is OC · sin θ ≒ Zc −Zd satisfies the relational expression (1) substantially. For this reason, the image sensor 10
The rotation angle θ of the optical axis of the eyeball can be obtained by detecting the position of each singular point (the corneal reflection image d and the end portions a and b of the iris) projected on the surface 9. At this time, the expression (1) is rewritten as β · OC · sin θ ≒ ｛(Za + Zb) / 2｝ −Zd (2) Here, β is the position d where the corneal reflection image is generated.
The magnification is determined by the distance L between the light-receiving lens 7 and the distance Lo between the light-receiving lens 107 and the image sensor 109, and usually has a substantially constant value.

According to the above-described principle, the direction of the line of sight of the examinee's eye (point of sight) can be detected.

Next, the operation of the photographing apparatus according to the first embodiment of the present invention will be described with reference to the flowchart of FIG.
Explanation will be given based on FIG.

In step 101, subject information is fetched from the image sensor 2 by the subject extraction circuit 3 as described above (see FIG. 5A). In the next step 102, whether the number of subjects is one or more? Is determined. As a result, if there is only one, the subject region has already been determined, and the process proceeds to step 108. If there are two or more, the process proceeds to step 103 to determine the main subject. In this embodiment, as shown in FIG.
Therefore, the process proceeds from step 102 to step 103. In step 103, for example, a region of the subject is set from information on the blur width, and an image corresponding to the frame of the region is generated by the subject frame generation circuit 4 based on the region information (FIG. 5B).
reference). Then, in step 104, this image (subject frame) is supplied to the finder control circuit 5, where the subject frame and the image pickup signal are combined, and the combined image is displayed on the display device 6.

At this time, the photographer observes the image on the display device 6 such as the electronic view finder through the finder lens 7, and the movement of the photographer's eyeball at that time passes through the above-mentioned visual axis detection optical system 8. The position of the line of sight is detected by the line-of-sight extraction circuit 11 based on the detected image of the eyeball detected by the detector 9.
Step 1 when the subject extraction switch is turned on
In step 06, the line-of-sight information generated by the line-of-sight extraction circuit 11 is supplied to the subject information control circuit 12.
In step 107, a main subject is determined by the subject information control circuit 12 from the line-of-sight information and the area information from the subject extraction circuit 3 (see FIG. 5C), and the subject area is determined.

After the subject area is determined by the subject information control circuit 12 as described above, the flow advances to step 108 to generate main subject information. Then, the next step 109
, The photographing lens 1 is driven by the photographing lens drive circuit 14 based on the main subject information, and the image pickup device 2 is driven by the image pickup device drive circuit 15.

When the main subject moves from the extracted position as shown in FIG. 6 (a), selection is made to follow based on the area information from the subject extraction circuit 3 (see FIG. 6 (b)).
At the same time, the driving of the photographing lens 1 by the photographing lens drive circuit 14 and the driving of the image pickup device 2 by the image pickup device drive circuit 15 are continued based on the main subject information obtained by following.

In the case where there are two or more subjects, the selection is made with the subject extraction switch. Instead, the gaze position is measured, and the position with the highest frequency is photographed. Is automatically determined to be the position where the user is gazing most, and the position is determined as the line-of-sight information by the subject information control circuit 1.
2 may automatically determine the main subject. Also, the information of the imaging signal used for extracting the subject may be a luminance signal or a color signal instead of the blur width,
These may be used together.

(Second Embodiment) FIG. 7 is a block diagram showing a photographing apparatus according to a second embodiment of the present invention, and the same parts as those in FIG. 1 are denoted by the same reference numerals.

In FIG. 7, reference numeral 17 denotes a memory for temporarily storing a subject frame.

In the second embodiment, at the time of subject extraction, the focus lens group of the photographing lens 1 is driven to extract each subject located at a different subject distance from the image signal, and display the subject together. This allows the photographer to determine the main subject from the frame.

Next, the operation of the photographing apparatus according to the second embodiment of the present invention will be described with reference to the flowchart of FIG.
A description will be given based on FIG.

When the photographer is ready to extract a subject, in step 201, the photographing lens driving circuit 14
The focus lens in the photographing lens 1 is set at the infinity position through the process, and in the next step 202, the subject information is fetched from the image sensor 2 by the subject extracting circuit 3 (see the left diagram in FIG. 9A). In step 203, similarly to the first embodiment, an area of the subject is set based on, for example, information on the blur width, and an image corresponding to a frame of the area is generated by the subject frame generation circuit 4 based on the area information. (See the right figure in FIG. 9 (a)). And step 2
At 04, this image (subject frame) is stored in the memory 17. Next, the process proceeds to step 205, where the previous focus lens is driven to the nearest side by a predetermined amount, the subject distance is changed, and it is determined in step 206 whether the same operation has been repeated a predetermined number of times. .

If the operation has not been performed the predetermined number of times, the operation returns to step 201, and the same operation is repeated. In this embodiment, as shown in FIG.
(d) is assumed to be performed five times. Therefore, the subject frames extracted in each loop are sequentially stored in the memory 17.

After the focus lens is driven a predetermined number of times to generate a subject frame for a predetermined number of times, the operation proceeds to step 207, where the subject frames stored in the memory 17 are combined, and the next step is performed. At 208,
The combined image of the subject frame is supplied to the finder control circuit 5, where it is combined with the video signal and displayed on the display device 6 (see FIG. 10A).

At this time, the photographer observes the image on the display device 6 such as the electronic view finder through the finder lens 7, and the movement of the photographer's eyeball at that time is, as described above, the visual axis detection optical system. Detected by the detector 9 through 8,
The gaze position is detected by the gaze extraction circuit 11 based on the detected eyeball image. In step 209, the subject extraction switch is turned on to proceed to step 210, where the gaze extraction circuit 11 Is supplied to the subject information control circuit 12. In step 211, a main subject is determined by the subject information control circuit 12 from the line-of-sight information and the area information from the subject extraction circuit 3 (see FIG. 10B), and the subject area is determined.

After the subject area is determined by the subject information control circuit 12 as described above, the flow advances to step 212 to generate main subject information. Then, the next step 109
, The photographing lens 1 is driven by the photographing lens drive circuit 14 based on the main subject information, and the image pickup device 2 is driven by the image pickup device drive circuit 15.

When the main subject moves from the extracted position, it is selected to follow based on the area information from the subject extracting circuit 3, and the photographing is performed based on the main subject information obtained by following. The driving of the photographing lens 1 by the lens driving circuit 14 and the driving of the imaging element 2 by the imaging element driving circuit 15 are continued.

As in the first embodiment, as a method of selecting a main subject from among a plurality of objects, a position where a gaze is gazed is measured, and a position with a high frequency of gaze is most closely watched by the photographer. The configuration may be such that the position is automatically determined and the position is regarded as the line-of-sight information, and the object information control circuit 12 automatically determines the main object. Further, even when the imaging signal for extracting the subject is not a blur width but a luminance signal or a color signal, the contrast is improved when the subject is in focus, so that the subject can be extracted by the same method.

(Third Embodiment) FIG. 11 is a block diagram showing a photographing apparatus according to a third embodiment of the present invention.
The same parts as those in FIG. 7 are denoted by the same reference numerals.

In the third embodiment, an area of a subject is extracted based on a video signal of a part of the screen that the photographer gazes at, and FIG. 13 is used in accordance with the flowchart of FIG. The operation will be described.

When the photographer is ready to extract a subject, in step 301, the subject information control circuit 12
The finder control circuit 5 is instructed to display the position of the line of sight on the display device 6. With the operation of the finder control circuit 5, the cursor is synthesized at the position based on the line of sight information from the line of sight extraction circuit 11 and displayed. It is displayed on the device 6 (see FIGS. 13A to 13B). Next Step 30
In step 2, it is determined whether or not the subject extraction switch is ON. If the switch is ON, the process proceeds to step 303, where the previous display is stopped. In the next step 304, the line-of-sight information when the object extraction switch is turned on is detected by the object extraction circuit 3, and in step 305, the line-of-sight information, that is,
The area of the subject is determined based on the video signal of the part watched by the photographer.
A region is extracted, and in a succeeding step 306, it is determined whether or not the extraction of the subject is successful. If the extraction of the subject is not successful, the process returns to step 301 to perform the same operation. If the extraction is successful, the process proceeds to step 307.

In step 307, a subject area is set based on the image signal near the line of sight, and this area information is supplied to the subject information control circuit 12. In the next step 308, the subject information control circuit 12 13 (c), subject area information is generated based on
), The photographing lens drive circuit 14 and the image sensor drive circuit 1
5 to be driven.

When the main subject moves from the extracted position, the main subject is selected to follow based on the area information from the subject extracting circuit 3 and is photographed based on the main subject information obtained by following. The driving of the photographing lens 1 by the lens driving circuit 14 and the driving of the imaging element 2 by the imaging element driving circuit 15 are continued.

According to each of the above embodiments, when there are a plurality of subjects from the image information, each region is set, and the region of the main subject is selected from the regions based on the gaze information of the photographer. Therefore, it is possible to extract a subject area intended by the photographer in real time without performing any special operation, in other words, an area to be a target of shooting control. A subject that is not focused will not be in focus or will not be improperly exposed.

(Modification) In each of the above embodiments, the line of sight information from the line of sight detection device is used to select the main object region from a plurality of object regions. However, the present invention is not limited to this. Alternatively, the cursor position displayed on the screen may be changed by an external dial operation or the like.

[0043]

According to the first aspect of the present invention, as described above.
According to the present invention, an image sensor that outputs an image signal is provided.
And extracting the area where the object exists based on the image signal.
Extraction means for extracting, and an area extracted by the extraction means.
Region selecting means for selecting at least one of the regions
Thus, the area extracted by the extraction means is duplicated.
If there are, the intended area can be selected.

According to the second aspect of the present invention,
An image sensor that outputs an image signal;
Extracting means for extracting an area where the object exists based on the
Gaze detection means for detecting gaze, and gaze of the gaze detection means
Based on the detection result, the area extracted by the extraction means
Area selecting means for selecting at least one of the areas
By having, the area extracted by the extraction means
If there is more than one, you can select the intended area
You.

[Brief description of the drawings]

FIG. 1 is a block diagram of a photographing apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram for explaining an operation in the subject extraction circuit of FIG. 1;

3 is a diagram for explaining to For additional details about visual axis detecting method in the visual axis detection apparatus of FIG.

FIG. 4 is a flowchart illustrating an operation of a main part of the photographing apparatus according to the first embodiment of the present invention.

FIG. 5 is a diagram to help explain the operation of FIG. 4;

6 is a diagram for assisting the explanation of the operation of FIG. 4;

FIG. 7 is a block diagram of a photographing apparatus according to a second embodiment of the present invention.

FIG. 8 is a flowchart illustrating an operation of a main part of the photographing apparatus according to the second embodiment of the present invention.

FIG. 9 is a diagram for helping to explain the operation of FIG. 8;

FIG. 10 is a diagram for assisting the explanation of the operation of FIG. 8;

FIG. 11 is a block diagram of a photographing apparatus according to a second embodiment of the present invention.

FIG. 12 is a flowchart illustrating an operation of a main part of the photographing apparatus according to a third embodiment of the present invention.

FIG. 13 is a diagram to help explain the operation of FIG. 11;

[Description of sign]

REFERENCE SIGNS LIST 1 photographing lens 2 image pickup device 3 subject extraction circuit 4 subject frame generation circuit 5 finder control circuit 8 visual axis detection optical system 9 detector 11 visual axis extraction circuit 12 subject information control circuit 13 video signal conversion circuit 17 memory

Claims (3)

(57) [Claims] An image sensor for outputting an image signal;
Extraction to extract the area where the object exists based on the image signal
Output means and a small number of areas extracted by the extraction means.
At least one area selecting means for selecting one.
Characteristic optical equipment. 2. An image sensor for outputting an image signal;
Extraction to extract the area where the object exists based on the image signal
Output means, line-of-sight detecting means for detecting a line of sight, and line-of-sight detection
Extraction means based on the line-of-sight detection result of
Area selection for selecting at least one of the displayed areas
An optical device, comprising: 3. The method according to claim 2, wherein the extracting means corresponds to a size of the object.
3. An area to be extracted is extracted.
On-board optical equipment.