JP3414492B2 - Eye-gaze detection camera and printer - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [0001] BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to information from a photographer's eye.
(Gaze information and pupil size information), and
The image is converted to an image based on the information at the time of printing.
Eye-gaze detection camera and pudding to create beautiful photos
Data device. [0002] 2. Description of the Related Art Conventionally, input of various kinds of information to a camera is an example.
For example, it is performed by dial, button, etc., and the input information is
As the number increases, the operation environment becomes complicated. example
For example, the gaze direction of the photographer looking through the viewfinder is detected,
A technology for inputting information to a camera from line information is disclosed in
194237, JP-A-3-87818, etc.
More are disclosed. [0003] In the detection method, etc., the cornea, iris,
There is a method to detect using any of pupil, sclera (white eye) etc.
Many are disclosed (Japanese Patent Laid-Open No. 1-160537,
JP-A-2-206425). [0004] Further, the visual line information is recorded, and the visual line information is recorded.
Based on the gaze information position (main subject) when printing
A method of printing is disclosed (Japanese Patent Laid-Open No. 4-1343).
No. 30, JP-A-5-158132). Also,
Tone conversion processing is common in digital image processing
It has been done. [0005] As described above, image processing
Techniques for reproducing optimal images based on
You. However, the person in charge of creating the optimal image is the photographer
If it is different from the
And optimal processing cannot be performed. When there are a plurality of main subjects,
Trying to optimally process the gaze information by leaving it on the film
Provides the best image even if it is realized without performing gradation processing.
Cannot be offered. In addition, when you are watching
Depending on the situation, the image obtained may be different
Many. [0007] The present invention has been made in view of the above problems.
The purpose is to capture the gaze information of the photographer.
Optimally for post-processing based on the film information
Restoring a natural image. [0008] [MEANS FOR SOLVING THE PROBLEMS] To achieve the above object
The present inventionSight ofThe line detection camera detects the eye movement of the photographer.
Outgoing line-of-sight detecting means and detected by this line-of-sight detecting means
Based on the detected photographer's eye position information
Main subject detection means for detecting a subject of interest;Photographer's eyes
Pupil detection means for detecting the size of the hole;
Exposure for the main subject detected by the detection means
And information detected by the pupil detection means.
Pupil sizeInformation on film or fill
Information recording means for recording on a recording medium on Mpatrone,
And the subject image exposed on the film is
Image reading means of a printer device provided outside the camera.
Is converted to an image signal.
The gradation conversion means of the printer device records the information on the recording medium.
Properly print the main subject based on the recorded information.
Used to convert the gradation of the above image signal
BeIt is characterized by the following. [0009] [0010] [0011] [Function] That is, the present inventionSight ofIn line detection cameras, gaze detection
Means for detecting the movement of the photographer's eyes and detecting the main subject
Is the eye of the photographer detected by the gaze detecting means.
The main subject is repeatedly detected based on the position information,pupil
The detecting means detects the size of the pupil of the photographer, and
The column indicates the main object detected by the main object detecting means.
Information about the exposure of the object and the pupil detection hand
Information about the size of the pupil detected by the step
On a recording medium on the
You. And especially, the subject exposed on the film
The image is read by a printer installed outside the camera.
Means for converting the image signal
Is the above-mentioned recording in the gradation converting means of the above-mentioned printer device.
Based on the information recorded on the medium, the main subject can be properly
Converting the gradation of the image signal so that
Used for. [0012] [0013] [0014] DESCRIPTION OF THE PREFERRED EMBODIMENTS Before describing embodiments of the present invention,
Explanation of the principle of gaze detection employed in the embodiment of the present invention
I do. There are various methods for detecting the gaze direction.
However, here are some existing methods that can be applied to cameras.
The first pull well known to those skilled in the art
A corneal reflection image called a Kinue image and a reflection image of the fundus, or a rainbow
Briefly explain how to detect using the edge of sampling
You. Since the structural description is already known, it is omitted here.
I do. First, FIG. 2 shows a light beam projected substantially from the optical axis.
The state of the light reception output of the reflected light from the eye will be shown and described. Casually
When light is projected on the eyeball 90 shown in FIG.
The first Purkinje image with high light output has a weak reflected light amount and is a reflected image
It is difficult to detect because of the different positions where they can be performed. The fundus when light is projected on the eyeball 90
Is a rainbow in which the fundus image is the peripheral image of the pupil due to the reflected light 95b from
It is detected as a silhouette of the sampling edge 94. This fundus
Image 95b from the image is shown together with the first Purkinje image 95a.
As shown in Fig. 2, the gaze direction is detected using these two images.
Put out. Reference numeral 91 denotes a rainbow sample, 92 denotes a sclera (white eye), 9
3 denotes a pupil, and 94 denotes a rainbow sampling edge. FIG. 3 shows a change in the detected image due to the rotation of the eyeball.
The situation will be described and described. The optical axis 98 of the eyeball 90 and the eye
When the luminous flux is parallel, as shown in FIG.
The center of the fundus image 95b, ie, the center of the pupil, and the firstLeKinue
The center of the image 95a is coincident. And the eyeball 90ButTimes
When the optical axis 98 is turned, the optical axis 98 is shifted to the eye as shown in FIG.
The ball 90 rotates around the rotation center 90c. In this case, the center of the fundus image 95b is
Receives reflected light at different positions on the sensor pixel row
it can. Further, the center of the first Purkinje image 95a is located on the fundus.
Light is received at a position relatively different from the center of the image 95b. This
This is because the center of the curved surface in front of the cornea is the center of rotation of the eyeball.
Because it is different. Therefore, the pair of sensor pixels of these two images is
Between the absolute position and the relative displacement between the two images
And the amount of rotation and shift of the eyeball 90 of the photographer looking through the viewfinder.
To determine where the photographer is looking
Can be determined. In the present invention, the gaze detection image
At least the corneal reflection image 95a or the fundus reflection image 95b.
Detect using one. Next, in FIG. 4, the center of the eyeball is fixed.
Rotation and corneal reflection 95a and fundus reflection 95b
Will be described. Fundus reflex 9 if the center of the eyeball is fixed
5b, the center of gravity position ix of the corneal reflection 95a,
Rotation angle is detected by detecting the center of gravity including both or both
it can. In the figure, px is the center of gravity of the corneal reflection 95a.
Ix indicates the position of the center of gravity of the fundus reflection 95b. The fundus reflex shown in FIG.
This is a so-called “red-eye state” in which the reflected light is large. Bright condition
If the rainbow sampling is stopped down in the or the reflected light does not return to the light receiving system
If it is not the above red eye condition, the fundus reflection 95b
The output drops further. The view from the center of the viewfinder
In this case, FIG. 4A shows the center at a rotation angle of 0 (reference), and FIG.
Is a negative rotation angle and the left side is shown, and FIG.
Will be. Next, FIG.ofShift and corneal reflex 95
a and the state of the fundus reflection 95b will be described. Fundus reflex 9
5b, the center of gravity position ix of the corneal reflection 95a,
If the position of the center of gravity including both
The shift amount can be detected. In the figure, px is the weight of the corneal reflection 95a.
The center position, ix, indicates the position of the center of gravity of the fundus reflection 95b.
You. In addition, similarly to FIG.
Red-eye state, and fundus reflex 95 if not red-eye state
The output of b further decreases. The view from the center of the viewfinder
In the case shown in FIG. 5A, shift 0 (reference) is at the center, and FIG.
5 (b) shows the shift amount negative and the left side, and FIG. 5 (c) shows the shift amount positive.
You are looking to the right. Generally, one shutter cycle
Sequence, that is, 2nd release from near 1st release
By the time the eye shift does not change significantly
The relative movement and the reference position
You can know the direction to look. Also, the reference position
The detection is preferably performed in a sequence. Hereinafter, the present invention based on the above-described principle will be described.
An example will be described. First, FIG.
Of the eye-gaze detecting camera and the printer device according to the embodiment of the present invention
Will be described. In the eye-gaze detection camera,
The outputs of the output unit 1 and the pupil diameter detection unit 2 are input to the subject detection unit 3
And the output of the subject detection unit 3 is a film recording unit.
4 inputs. Then, the film recording section 4
The information is transmitted to the printer device via the film. one
On the other hand, in the printer device, the image reading unit 5 and the information reading unit
The output of the capture unit 6 is connected to the input of the image conversion unit 7,
The output of the image converter 7 is connected to the inputs of the display 8 and the output 9.
Have been. In such a configuration, first, the line-of-sight detection
On the camera side, the line-of-sight detection unit 1 is the photographer looking through the viewfinder.
And sends the information to the subject detection unit 3.
You. Then, the pupil diameter detector 2 determines the size of the pupil of the photographer.
And sends the information to the subject detection unit 3. Subject detection
The unit 3 includes at least one main unit based on the line-of-sight information from the line-of-sight detection unit 1.
The subject is extracted, and the size of the pupil from the pupil diameter detection unit 2 is adjusted.
Information about the corresponding pupil diameter from the related information
You. The film recording unit 4 is detected by the subject detection unit 3
Information related to one or more main subjects (corresponds to main subjects)
Information about the size of the pupil and other pupils
Information) Information on the size of the pupil to be recorded on film
(Magnetic recording, optical recording (barcode imprinting) recording)
You. In this way, the film is developed and printed.
When the image data is passed to the printer device, the printer device
The stripping unit 5 reads image information with a film scanner
The image information is sent to the image converter 7. The information reading unit 6 is
Information recorded on the film (optical recording until magnetic recording)
(Main subject information and pupil size detected by gaze
Information) and sends the main subject information to the image conversion unit 7.
You. The image converter 7 converts the image information, the main subject information, and the pupil.
Display an image whose gradation is converted from the size information on the display unit 8
(The image before conversion can also be displayed.)
Send information. Then, the output unit 9 confirms the converted image
Output later. Hereinafter, referring to the flowchart of FIG.
The sequence on the eye-gaze detecting camera side will be described. Eye-gaze detection
When the sequence on the camera side starts, the line-of-sight detection unit 1
Process information (gaze detection and one or more main subjects)
(Step S1), pupil diameter detecting section 2 detects pupil size
(Step S2). And the subject detection unit
3 performs a sequence related to AF and exposure (step S
3) In synchronization with film winding (or rewinding)
The film recording unit 4 detects one or more main
The subject information and the pupil size information are recorded (step S
4), this sequence ends (step S5). Next, referring to the flowchart of FIG.
The sequence on the counter side will be described. The sheet on the printer side
When the cans start, the information reading unit 6 turns the film,
One or more key features recorded on the processed film
The subject information and the pupil size are read (step S1).
1), the image reading unit 5 reads image information
(Step S12). And this read image information
Flashing, circles, etc.
(Step S13), and the image conversion unit 7
The image is subjected to gradation conversion processing and confirmation (step S1).
4) The display unit 8 displays the converted image (step)
S15). After this confirmation, an image is output from the output unit 9 (scan).
Step S16), end the present sequence (Step S16)
17). It is to be noted that, in the case of NG or the like, it is confirmed that the
The characteristics of the conversion may be changed. Further display unit 8
The image gradation of the output unit 9 may be adjusted. Fi
When the information recorded in the room is only the main subject information, 1
Scan images of more than one main subject
The gradation is determined from the density of each main subject position
In the case of only the size information, the pupil size change amount and the film
Depending on the density of each block (the screen is divided into multiple blocks)
The gradation may be determined in the same manner. Also a scanner
Scanner integration to ensure dynamic range
Scan multiple times at different times and correct by integration time
Processing may be performed using images. FIG. 8 shows a first embodiment of the present invention.
An eye-gaze detecting camera and a camera according to a second embodiment
The configuration of the printing apparatus is shown and described. The second embodiment
Is realized by one-dimensional gaze detection, but two-dimensionally
Of course, if detection is performed, the accuracy is further improved. As shown in FIG.
On the camera side, the output of the photometric unit 23 is the KCPU 21 (camera side).
CPU) and the I / F circuit 22 is connected to the LED
Connected to the road 28, the line sensor 24, and the KCPU 21;
The LED circuit 28 includes the I / F circuit 22 and the finder optical system 2.
5 is connected. Further, the line sensor 24 has an I /
F circuit 22, connected to finder optical system 25,
The optical system 25 includes an LED circuit 28 and a line sensor 24.
It is optically connected and the photographer's eyes and the shooting scene
Optically connected. And hoist and rewind
The circuit 26 and the magnetic circuit 27 are composed of the KCPU 21 and the film 29.
It is connected to the. On the other hand, on the printer side, a magnetic reading circuit
32 is connected to the film 30 (developed),ILum
The scanner 33 includes the film 30 and the PCPU 31 (printer).
Side CPU) and the image memory 34. Display circuit
Reference numeral 35 is connected to the image memory 34 and the PCPU 31. Out
The input image memory 36 is connected to the image memory 34 and the PCPU 31.
It is connected to the linter unit 37. It should be noted that AF is known.
Multi AF (for example, scan AF: JP-A-4-3248)
07 publication, etc.). With such a configuration, on the camera side,
The photometric unit 23 sends photometric information to the KCPU 22. Gaze detection
In the block of I / F circuit 22, LED circuit 28 and line
Controls the sensor 24 and communicates with the KCPU 21
Communication is performed, and the LED circuit 28 is connected to the finder optical system 25.
The light beam is projected to the eye through a part, and the line sensor 24
Of reflected light from the camera through the viewfinder optical system 25
No. The KCPU 22 reads the image information of the eye from the line-of-sight detection block.
Extract information on gaze direction and pupil size from the report,
Extract one or more information about the main subject from the information
With pupil size information, photometric information, and exposure information
Transmit to circuit 27 to control winding / rewinding circuit 26
Then, it is recorded on the film 29 by a magnetic signal. On the other hand, the magnetic reading on the printer side is performed.
The take-off device 32 is magnetically applied to the film 30 from the film 30.
The information from the camera recorded in the
Send to PU31. The film scanner 33 is a PCPU 31
Control signal (optimal scanning speed based on photometric information and exposure information)
Read the image information from the film 30
The image is sent to the image memory 34. PCPU 31 is the main subject information
And sends the main subject information to the display circuit 35 based on the image.
And gradation conversion based on information from the magnetic reader
(Table conversion, function conversion) and display the processed image
The output image memory 34
Send to The display circuit 35 displays the image stored in the image memory 34.
Image table based on image and main subject information from PCPU 31
Display and main subject display. The output image memory 36 is newly
Store the created image. Printer device 37 created
And output the image. FIG. 9 shows the details of the visual axis detection optical system.
The configuration is shown and described. FIG. 9A shows the optical arrangement.
You. The detection sensor uses an accumulation type line sensor to detect
Detect reflected light. The light source of LED0, LED1
LED0 has a red-eye image on the detection line sensor
LED1 is on the detection line sensor
So that the red-eye image does not occur
To emit light at an angle of 2 to 5 degrees. This
The light emitted from LED0 and LED1 is a beam splitter.
The light is projected to the eye through the
Guided to the line-of-sight optical system and line sensor via the muscle splitter
It is. FIGS. 9B and 9C show LED0 and LED1.
3 shows an image of an eye when the light is projected. For details, see FIG.
(B) shows the light emitted from LED0, which shows the Purkinje image and the red-eye image.
FIG. 9 (c) shows the detected state.
There is a Purkinje image but no red-eye image can be detected
It shows the situation. It should be noted that the process performed by LED0 and LED1
The position of the Lukinje image can be the same in the x-axis direction.
LED0 and LED1 are arranged as follows (on the detection sensor
Position the light source at a position orthogonal to the direction of detection of the Purkinje image.
Arrangement). Next, FIG. 10 shows the lines of FIGS. 9 (b) and 9 (c).
The signal on the sensor is shown and described. The image in FIG.
The difference processing between the signal in FIG. 10A and the signal in FIG.
Shows the results of the experiment, where only the red-eye image is extracted
I understand. Therefore, from the image shown in FIG.
Red-eye center of gravity (pupil center of gravity) position and maximum value (Purkinje image)
Position) is detected, and the gaze direction can be determined.
The size of the pupil can be detected at both edges. Hereinafter, referring to the flowchart of FIG.
The sequence on the eye-gaze detecting camera side in the second embodiment is
explain. Camera sequenceToWhen started, the sequence
Initialize. That is, here, the variable i is set to 0.
SetOSet the focus point to the center (Step
S21). Subsequently, the first release is determined (the
Step S22). Then, the first release switch is turned off.
In the case of, the process proceeds to step S31, and the first release
If the switch is ON, the subroutine "line-of-sight detection 1"
To perform gaze detection synchronized with the 1st release.
(Step S23). That is, as shown in the flowchart of FIG.
As described above, when the subroutine "line-of-sight detection 1" is started,
The subroutine "line-of-sight detection" described above is executed (step S
32) store the line-of-sight block data in the memory E (i),
The size is stored in the memory P (i) (step S33),
The process returns to the main routine (step S34). Subsequently, a subroutine "line-of-sight detection 2" is executed.
Then, continuous gaze detection is performed (step S24). Immediately
That is, as shown in the flowchart of FIG.
When the routine “Gaze detection 2” starts, the variable i is incremented.
(Step S35), the subroutine described later is executed.
Then, a "line-of-sight detection" is executed (step S36). Line of sight
Memory E (i) and the pupil size in memory P
(I) (step S37), and
The process returns (step S38). Next, the KCPU 21 again executes the first
Is determined (step S25), and the first release is performed.
If the switch is OFF, the process proceeds to step S31,
2nd release when the 1st release switch is ON
Is determined (step S26). And 2nd
If the release switch is OFF, return to step S24.
Multi-speed when the 2nd release switch is ON.
Performs can ranging. Here, the subroutine "
The subroutine "measurement" is performed according to the information detected in "detection 1".
Distance ”to set a distance measuring point (step S2).
7). That is, as shown in the flowchart of FIG.
When the subroutine "ranging" starts, multi-ranging
(Step S121), and the evaluation of the data E (0) is performed.
Perform (Step S122). And E (0) = 0
In this case, a known evaluation AF method that does not use the line-of-sight data
The AF point is set by the algorithm (step S12
3) If E (0) = 0, distance measurement of E (0) data
Set the AF point based on the block information (step
S124). In addition, the lens
Is calculated (step S125), and this sequence
And returns to the main routine (step S126). Next, the KCPU 21 executes the shooting sequence
(Lens drive, exposure determination, exposure) (step S2).
8) Winding starts (step S29), and winding starts
Information is synchronously recorded on the film 29 (step S3).
0), this sequence ends (step S31). The subroutine executed in step S30
The sequence of the chin "information recording" is shown in FIG. Immediately
When the subroutine "information recording" is started,
Eye movement detection from release to 2nd release
From data E (i), histogram processing for each block
Normalization by the number of detections is performed (step S81), and a predetermined value is obtained.
Select the top three blocks from the above blocks and
Information is output (step S82). And the corresponding
The pupil size information of the lock is output (step S8).
3) Output other pupil size information (step S)
84), and exits this sequence (step S85). Next, referring to the flowchart of FIG.
The sequence of the subroutine "line-of-sight detection" will be described.
When the subroutine "line-of-sight detection" starts,
(Step S41), and reset the sensor.
LED0 is projected (step S42), and the sensor signal D0 is emitted.
Is detected (step S43). Next, reset the sensor.
The LED 1 is emitted (step S44), and
The sub signal D1 is detected (step S45). And this
Signal processing to detect the line of sight with the detection signals D0 and D1
Exit (step S46), exit this sequence (step S46).
Step S47). In step S45, the flow of FIG.
As shown in the chart, the subroutine "Signal processing"
Is started, the detected data D0 and D1 are given a predetermined weight.
Data D (D = a × D0−b × D1; a,
b predetermined variable) is created (step S51). Soshi
The center of gravity xd is detected from the data D (step S5).
2) From the data D0, the MAX value position (when there are multiple MAX values)
If this is the case, the position of the center of gravity of the MAX data group) x0 and the MAX value M
0 is detected (step S53), and MAX is obtained from the data D1.
Value position (If there are multiple MAX values,
The center position x1 and the MAX value M1 are detected (step S5).
4). Subsequently, the evaluation of the amount of displacement between x0 and x1 (eye
The motion amount is evaluated) (step S55). And |
When x0−x1 | <ε is not satisfied, the process proceeds to step S62.
When | x0−x1 | <ε, the average is calculated from the data D0.
The value Av0 is obtained (step S56). Then the data
The difference between the MAX value M0 of D0 and the average value Av0 is evaluated (k is
(Predetermined value) (step S57), and M0−Av0> k is not satisfied.
If not, the process proceeds to step S62, where M0−Av0> k
In the case of, the average value Av1 is obtained from the data D1 (step
Step S58). Subsequently, the maximum value M1 of the data D1 is
The difference of the average value Av1 is evaluated (k is a predetermined value) (Step S
59) If not M1-Av1> k, step S6
2 and if M1-Av1> k, the detection signal D
The pupil diameter dp is detected (step S60). The subroutine executed in step S60
The sequence of the chin “pupil diameter detection” is shown in FIG.
It is as expected. That is, the subroutine "pupil diameter detection"
When started, the signal of the detection data D is binary-coded at a predetermined output.
(Step S71), and as shown in FIG.
Detect change positions xf and xg of binary data in the x-axis direction
(Step S72). Then, the size of the pupil is measured (d
p = xg-xf) (step S73), and this sequence
Is exited (step S74). Returning to the sequence of FIG.
The position of the center of gravity of the red eye xd and the position of the Purkinje image x0
(Detection block is 1 to N)
Step S61). On the other hand, in step S62, the detection block
To 0 (non-existent block No.)
Exit (step S63). In step S55, the detection signals D0,
The determination of the eye movement in D1 is performed, and step S5
7. In S59, a blink determination is made. Also,
When there is a nozzle signal source such as internal reflection in the viewfinder optical system
If the eyepiece side of the viewfinder is shielded in advance, each LED is turned on
Nozzle signals Dd0 and Dd1
Dd = a × Dd0−b × Dd1 (where a and b are predetermined
), And the difference between the detection signal and the noise signal Dd is added to the signal D.
It is good to store. FIG. 18 shows the configuration of the line sensor.
Will be described. FIG. 18A shows the line sensor block.
Fig. 2 shows a block diagram of the configuration, where BS is composed of photoelectric conversion elements.
Have been. The DBS shields the BS to compensate for dark current.
This is a basic block for regular use. In addition, the shift register
Each BS is driven and controlled by a control signal from the KCPU 21.
You. Each BS receives control signals (T), (C) and
And the integral control by the monitor signal (M) output by the BS and the BS itself.
It is. Integral control detects peak value (any BS has
Outputs a signal that terminates integration when the bell is reached) and time
Perform control. The signal (M) is recorded in the KCPU 21 and the signal
(T) is held until the next integration starts, and C
Hold. Furthermore, the image of the eye is converted to the signal of the shift register.
Accordingly, after the dark current difference is erased and amplified, the A /
It is sent to the D circuit. And the integration time control is the brightest
Determined by the state (when red-eye occurs) and integrated when no red-eye occurs
The time is the integration time when red eye occurs and is determined by the control signal (T).
It is. FIG. 18B shows the internal circuit of the BS.
FIG. Gauge for photoelectric conversion by PD and integral control
Stored in the capacitor C via the gate G0
1 resets PD and C together with G02. Gate G
03 performs read amplification, and the gate G04 is a shift register.
The signal is read according to the signal of the star. And Daioh
Step D01 performs peak value detection. Hereinafter, referring to the flowchart of FIG.
The sequence of the integration time control will be described. Integration sequence
Is started, the sensor and timer are reset (step
S101), the light emitting LED0 is determined (step S1).
02). In the case of LED0 light emission, the same as the integration start
At this time, a timer is started (step S103). Subsequently, the timer t is determined (step
S104) When t> td (td is a predetermined limit value)
If not, go to step S108 and if t> td is not satisfied
Judge the integration monitor (peak value detection) signal.
Step S105). When the monitor signal is OFF, the step
Returning to step S104, the integration is continued. Monitor signal ON
In the case of, the integration ends (step S106), and the integration time
tc is recorded by the KCPU 21 (step S107).
Is read out (step S113), and the process returns.
(Step S114). In the above step S102, LED0
If is not emitted, the timer
-Start (step S109). And the integration module
(Peak value detection: monitor determination level is step S1
(Set higher than 07) (Step S)
110), if the monitor signal is ON, step S108
And the timer t is determined when the monitor signal is OFF.
Is performed (step S111). Furthermore, t> tc is not satisfied
In this case (step S111), the process returns to step S110.
Continue the integration. On the other hand, in step S111, when t> tc
In this case, the integration is terminated, and the process proceeds to step S113.
(Steps S111 and S112). Step S108 above
If the signal is unreliable (integration time
Long: no eye, integration end is shorter than expected: eye
There is a strong reflection other than
The process proceeds to step S113 (step S108). The time of each integration is determined and
If the integration time is not within the time, the reliability is low.
The determination may be made to invalidate the data. Next, FIG.
The chart shows the sequence on the printer side and explains it.
You. When the printer sequence starts, the PCPU
31 initializes the data (step S20)
1) Information recorded magnetically by the magnetic reading circuit 32
Is read (step S202). And films
The canner 33 reads the image information and stores it in the image memory 34.
(Step S203), the display circuit 35 displays the information based on the information.
Then, information and an image are displayed (step S204).
Further, the PCPU 31 converts the image (Step S).
205), and display the converted image (step S20)
6). Then, the image is confirmed (step S20).
8) In the case of NG, change the parameters and
Returning to step S204 (step S207), if OK
Outputs the converted image (step S209),
The process exits the cans (step S210). Next, the flowcharts of FIGS.
Refer to the above subroutine "image conversion" sequence
Will be described. When image conversion starts, the main subject position
Average output value of image block (K
0, K1, K2) and the average output value (K
a) is detected (step S301). Next,
The average value of the pupil at the position of the object (P0, P
1, P2) (step S302).
The average value (Pa) of the pupil diameter is detected (step S30).
3). Subsequently, the output difference between the main subject positions is detected.
(Step S304), K0−K2 <Kd (Kd is a predetermined
) (The contrast between the main subjects is large)
It moves to step S313. On the other hand, when K0−K2 <Kd,
Detects a change in pupil diameter (step S305), and P0−
If P2 <Pd (Pd is not a predetermined value), the dark portion
And the conversion to expand the range of the bright part (FIG. 24 (a))
(See (4)) (step S311). This step
In step S305, if P0−P2 <Pd, the pupil flatness
Comparison with the average size Pa is performed (step S306, step S306).
Step S307). Then, P0−Pa ≧ Pv (Pv is a predetermined value)
Value), the average value of the block of the first main subject
So that the force value is slightly below the center of the conversion output range.
(See (3) of FIG. 24A) (step S31).
0). Further, in the case of -Pv> P0-Pa> Pv,
The average output value of the block of the first main subject is converted output level.
24 (a) (see (1) of FIG. 24 (a)).
(Step S308). Also, -Pv ≦ P0-
In the case of Pa, the average output of the block of the first main subject
Convert so that the value is slightly above the center of the conversion output range
(See (2) of FIG. 24A) (step S30).
9). On the other hand, the steps from step S304
In S313, a change in pupil diameter is detected. Soshi
If P0−P2 <Pd (Pd is a predetermined value),
Put block P0 within conversion output range and dark area
And the conversion to expand the range of the bright part (FIG. 24 (b))
(See (8)) (step S319). Furthermore, P
When 0-P2 <Pd, the average pupil size Pa
Are compared (steps S314 and S31).
5). When P0−Pa ≧ Pv (Pv is a predetermined value),
In this case, the block of K0 is put in the conversion output range, and
Conversion to expand the range of the bright part (see FIG. 24B)
(See (7)) (step S318). Furthermore,-
If Pv> P0−Pa> Pv, change the block of K0.
Conversion into the conversion output range (see (5) of FIG. 24 (b))
Is performed (step S316). Also, -Pv≤P0-P
In the case of a, the block of K0 is put in the conversion output range.
In addition, the conversion for expanding the range of the dark part (FIG. 24B)
(See (6)) (step S317). In this way
A turn is made (step S312). FIG. 24 shows a conversion curve.
The numbers correspond to the numbers of the conversion methods in FIGS. Soshi
For conversion, the input and output may be converted and output in a table.
No. The conversion output range is determined by the output medium. The embodiments of the present invention have been described above.
However, the present invention is not limited to this, and various improvements and modifications are possible.
Of course, further modifications are possible. For example, gaze detection
Formula is Purkinje image position from one screen using area sensor
And the method of extracting the pupil edge may be used.
The pupil center of gravity and the Purkinje image are time-shared using the D sensor
An extraction method may be used. In the case of PSD, the pupil size is large.
The magnitude is estimated based on the light amount after the difference. In addition, reliability judgment
It is done by blinking and eye movements, but the camera and face contact
Tactile (pressing the eye against the camera finder)
It is even better to combine eye drop determination. Furthermore, gaze
Detection is performed in parallel with the sequence of cameras such as AF and AE
It may be. In extracting the main subject,
Block where the line stays continuously for a long time (line of sight)
May be selected. That is, the flowchart of FIG.
After initialization, as shown in the
In step S401, the variable i is incremented (step S4).
02), the above-described subroutine "line-of-sight detection" sequence
And when E (i-1) = E (i) is not satisfied,
(Step S404) MAX1 <E (i-1)
In this case, J (E (i-1)) is stored in MAX1.
(Step S410), E (i-1) is set to MAX1.
Tore (step S411), and proceed to step S412
I do. Then, in step S409, MA
When X1 <E (i−1) is not satisfied, MAX2 <E
If it is (i-1) (step S414), MA
Store j (E (i-1)) in X2 (step S41)
5) Store E (i-1) in MAX2 (step
S416), and proceeds to step S412. Further, in step S414, MA
When X2 <E (i−1) is not satisfied, MAX3 <E
In the case of (i-1), MAX (j (E (i−
1)) is stored (step S415) and stored in MAX3.
E (i-1) is stored (step S416), and
The process moves to step S412. In step S417, M
If AX3 <E (i-1) is not satisfied, step S41 is executed.
Move to 2. In step S 412, J (E (i−
1)) is set to 0, and J (E (i)) = J (E (i)) + 1
(Step S405), the first release switch
If the switch is OFF, the sequence ends (step
S413). And 1st release switch and 2n
Return if d-release switch is on
(Step S408). On the other hand, the line of sight at the time of the first release
(Line of sight) and the block at the second release (line of sight)
Weight) (one layer, the main subject)
Is often closely related to: Increase the number of times or time
Evaluation) may be performed. Furthermore, in the AF method,
Passive two-dimensional wide field of view using area sensor
AF may be used, and a small number of discrete multi-A
F may be used. In addition, the image conversion method uses gaze information and pupil
Of course, any other method that uses hole size information can be used.
Good. Further, in the subroutine "image conversion",
Therefore, as shown in FIG.
It is good to have a difference (an example of the conversion characteristic is shown in FIG. 27).
). That is, as shown in the flowchart of FIG.
In the main subject pupil diameter detection (P0, P1, P2, Pa)
Is performed (step S501), and P0−P2 <Pd is satisfied.
In the case, P0−P2 <Pd
27 is a change characteristic indicated by a broken line from a normal characteristic.
And returns (steps S502 to S5).
04). The conversion method is changed according to the absolute value of the pupil.
You may. Furthermore, when only the main subject information from the line of sight is
As shown in FIG. 28, each block corresponding to the main subject information
The conversion may be performed with the average output of the
(An example of the conversion characteristic is shown in FIG. 29). That is, as shown in the flowchart of FIG.
As described above, the main subject block output detection (K0, K1, K
2) is performed, the average output Ka is detected, and K0 to K2 are set to M0.
To M2, Ma (Steps S601 to S60)
3). Then, comparison between M0, M2 and Mmax, Mmin
And based on the comparison result, the characteristic shown in FIG.
(1) to (6) are selected (Steps S605 to S6)
15). In addition, the information recorded on the film is
It is more preferable to determine the contents by adding a header signal. For example
(First line of sight block header signal) (Block position)
(Average pupil diameter) (Second line-of-sight block header signal)
Lock position) (Average pupil diameter) (Third line-of-sight block header
-Signal) (block position) (average pupil diameter) (pupil diameter head)
(Pupil diameter 1) (pupil diameter 2) ... (end signal)
It is. As described in detail above, according to the present invention, the current
Main subject extracted from line-of-sight information in camera configuration
Gradient from body and pupil size information (only one may be used)
An ordered image can be obtained. According to the above embodiment of the present invention, the following:
Such a configuration is obtained. (1) A camera that detects the movement of the photographer's eyes with a camera
The line detection device and the detected eye position information for a predetermined period are mainly used.
Object detection device for selecting the subject of interest
A film recording device that records subject information,
A printing device that performs gradation conversion based on information.
A camera system, characterized in that: (2) In the above (1), the gaze detecting device reflects the light from the pupil.
The gaze direction is detected from the light and the reflected light from the cornea. (3) In the above (1), the gaze detection device reflects from the pupil.
The gaze movement is detected from the light. (4) In the above (1), the line-of-sight detection device detects the reflected light from the cornea.
Gaze movement is detected. (5) In the above (1), the subject detection device can
The main subject is detected from the position where the number of times of detection is large. (6) In the above (1), the subject detection device can
The main subject is detected from the position where the continuous stay time is long. (7) In the above (1), the subject detection device performs
To detect one or more main subjects. (8) Vision for detecting the movement of the photographer's eye in the camera
The line detector and the eye position information for a predetermined period detected are mainly used.
A subject detection device that selects the subject of interest and the pupil of the photographer's eye
Pupil detection device that detects the size of the hole
A film recording device for recording subject information and pupil information,
A camera system comprising: (9) In the above (8), the gaze detection device reflects the light from the pupil.
The gaze direction is detected from the light and the reflected light from the cornea. (10) In the above (8), the eye gaze detecting device performs the reaction from the pupil.
The movement of the line of sight is detected from the emitted light. (11) In the above (8), the line-of-sight detection device uses the reflected light of the cornea.
More gaze movement is detected. (12) In the above (8), the object detection device is set within a predetermined period.
Detects the main subject from the position where the number of detections is large. (13) In the above (8), the subject detecting device can
Detects a main subject from a position where the continuous stay time is long. (14) In the above (8), the subject detection device performs
At least one main subject is detected. (15) In the above (8), the pupil detection device is a gaze detection device
It is characterized by performing using the signal detected in. (16) The pupil detected by the pupil detection device in the above (8)
The size of is determined by the size of red eyes
You. (17) Camera detects movement of eye of photographer
Based on the gaze detection device and the detected eye position information for a predetermined period
A subject detection device that selects the main subject,
A pupil detection device that detects the size of the pupil and a pupil on the film
Film recording device for recording hole information and film information
And a printing device that performs gradation conversion based on
Camera system characterized by the above-mentioned. (18) In the above (17), the eye gaze detecting device detects
The gaze direction is detected from the reflected light and the reflected light from the cornea. (19) In the above (17), the eye-gaze detecting device detects
The movement of the line of sight is detected from the reflected light. (20) In the above (17), the line-of-sight detection device uses the corneal reflection.
The gaze movement is detected from the light. (21) In the above (17), the subject detection device operates for a predetermined period.
The main subject is detected from the position where the number of times of detection is large. (22) In the above (17), the subject detection device operates for a predetermined period.
Main subject is detected from the position where the continuous stay time is long
You. (23) In the above (17), the subject detection device operates for a predetermined period.
In, one or more main subjects are detected. (24) In the above (17), the pupil detection device is a gaze detection device.
The detection is performed using a signal detected by the device. (25) The pupil detected by the pupil detection device in (17).
The size of the hole is determined by the size of the red eye
You. (26) In the printing apparatus, a film (developed
Image reading device that reads the video of
Main subject information or pupil size information recorded in the
An information reading device for reading one or both of
An image that performs image gradation conversion based on information from an information reader
An image conversion device.
Tem. (27) Information recorded by the camera in (26) above
Is at least the location of the main subject or the pupil of the photographer
Contains one or both of the size information. (28) In the above (26), the image conversion device performs the conversion.
It is characterized in that an image can be confirmed. (29) In the above (26), the image conversion device records the characteristics.
It is characterized in that it is converted by a bull. (30) In the above (26), the image conversion apparatus determines the characteristics.
The conversion is performed by the following function. (31) a line-of-sight detection device that detects movement of the photographer's eyes;
Select the main subject from the detected eye position information for a predetermined period
Object detector and the size of the pupil of the photographer's eye
Pupil detection device to detect and detect multiple distance information of the screen
Multi-ranging device and main subject information and pupil information on film
And a film recording device for recording information.
And images of film (developed)
Image reading device and main subject recorded on film
Read one or both of body information and pupil size information
Information reading device to take and the information of said information reading device
And an image conversion device that performs gradation conversion of the image based on the
And a printing device.
Mela system. (32) In the above (31), the eye-gaze detecting device detects
The gaze direction is detected from the reflected light and the reflected light from the cornea. (33) In the above (31), the eye-gaze detecting device detects
The movement of the line of sight is detected from the reflected light. (34) In the above (31), the eye-gaze detecting device uses the corneal reflection.
The gaze movement is detected from the light. (35) In the above (31), the subject detection device operates for a predetermined period.
The main subject is detected from the position where the number of times of detection is large. (36) In the above (31), the subject detection device operates for a predetermined period.
Main subject is detected from the position where the continuous stay time is long
You. (37) In the above (31), the subject detection device operates for a predetermined period.
In, one or more main subjects are detected. (38) In the above (31), the pupil detection device is a gaze detection device.
The detection is performed using a signal detected by the device. (39) The pupil detected by the pupil detection device in (31).
The size of the hole is determined by the size of the red eye
You. (40) Information recorded by the camera in (31) above
Is at least the location of the main subject or the pupil of the photographer
Contains one or both of the size information. (41) In the above (31), the image conversion device performs the conversion.
It is characterized in that an image can be confirmed. (42) In the above (31), the image conversion apparatus records the characteristics.
It is characterized in that it is converted by a bull. (43) In the above (31), the image conversion apparatus determines the characteristics.
The conversion is performed by the following function. [0080] According to the present invention, the gaze information of the photographer can be obtained.Record
Recording mediumRecorded in therecoding mediaPost-processing based on the
Eye gaze detection turtle that restores an appropriate imageLaCan be provided
Wear.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing a configuration of a line-of-sight detection camera and a printer device according to a first embodiment of the present invention. FIG. 2 is a diagram showing a light receiving output of light reflected from an eye of a light beam projected from a substantially optical axis. FIG. 3 is a diagram illustrating a state of a change in a detected image due to rotation of an eyeball. FIG. 4 is a diagram showing a state of rotation, a corneal reflection 95a, and a fundus reflection 95b when the center of the eyeball is fixed. FIG. 5 shows a shift of the eyeball, a corneal reflex 95a and a fundus reflex 95.
It is a figure showing a situation of b. FIG. 6 is a flowchart illustrating an operation of the eye-gaze detecting camera according to the first embodiment. FIG. 7 is a flowchart illustrating an operation of the printer according to the first embodiment. FIG. 8 is a diagram illustrating a configuration of a line-of-sight detection camera and a printer device according to a second embodiment, which further embodies the first embodiment. FIG. 9 is a diagram showing a detailed configuration of a visual line detection optical system. FIG. 10 is a diagram showing signals on the line sensor in FIGS. 9 (b) and 9 (c). FIG. 11 is a flowchart illustrating the operation of the eye-gaze detecting camera according to the second embodiment. FIG. 12 is a flowchart illustrating a sequence of a subroutine “line-of-sight detection 1”. FIG. 13 is a flowchart showing a sequence of a subroutine “line-of-sight detection 2”. FIG. 14 is a flowchart showing a subroutine “line-of-sight detection” sequence. FIG. 15 is a flowchart illustrating a sequence of a subroutine “signal processing”. FIG. 16 is a flowchart showing a sequence of a subroutine “pupil diameter detection”. FIG. 17 is a flowchart showing a sequence of a subroutine “information recording”. FIG. 18 is a diagram illustrating a configuration of a line sensor. FIG. 19 is a flowchart showing a sequence of integration time control. FIG. 20 is a flowchart showing a subroutine “ranging” sequence; FIG. 21 is a flowchart illustrating an operation of the printer according to the second embodiment. FIG. 22 is a flowchart showing a sequence of a subroutine “image conversion”. FIG. 23 is a flowchart showing a sequence of a subroutine “image conversion”. FIG. 24 is a diagram showing a conversion curve according to the second embodiment. FIG. 25 is a flowchart showing a subroutine “line-of-sight detection” sequence according to an improved example. FIG. 26 is a flowchart showing a sequence of a subroutine “image conversion” according to an improved example. FIG. 27 is a diagram illustrating a state of conversion characteristics corresponding to FIG. 26; FIG. 28 is a flowchart showing a sequence of a subroutine “image conversion” according to an improved example. FIG. 29 is a diagram illustrating a state of conversion characteristics corresponding to FIG. 28; [Description of Signs] 1 ... gaze detection unit, 2 ... pupil diameter detection unit, 3 ... subject detection unit, 4 ... film recording unit, 5 ... image reading unit, 6 ... information reading unit, 7 ... image conversion unit, 8 ... Display unit 9, output unit.

Continuation of the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G03B 17/24 G02B 7/28 G03B 17/00 G03B 27/32

Claims (1)

(57) [Claims 1] A line of sight detecting means for detecting the movement of the photographer's eyes, and a main subject is repeated based on position information of the photographer's eyes detected by the line of sight detecting means. Main subject detecting means for detecting the size of the pupil of the photographer, and the main subject detected by the main subject detecting means
Information on the exposure of the pupil,
Information recording means for recording information on the detected pupil size on a film or a recording medium on a film cartridge, and the subject image exposed on the film is provided outside the camera.
The image is read by the image reading means of the printer provided.
Is converted into a signal, the image signal, the gradation conversion unit of the printer device
Based on the information recorded on the recording medium
Make sure that the image signal is
A gaze detection camera, which is used for converting gradation .