JPH07319037A - Visual-line detecting camera and printer device - Google Patents

Abstract

PURPOSE:To provide a visual-line detecting camera and a printer device which record information about the visual line of a photographer on film and restore an optimum image through subsequent processing according to film information. CONSTITUTION:A visual line detecting portion 1 detects the movement of a photographer's eye, a subject detecting portion 3 repeatedly detects main subjects for photography according to eye position information, and a film recording portion 4 records information about the plurality of main subjects for photography on film. An information reading portion 6 reads the information about the plurality of main subjects for photography from a storage medium, an image reading portion 5 converts an image of each subject for photography exposed on the film into an image signal, an image converting portion 7 converts the gradient of the image signal so that the plurality of main subjects for photography are properly printed, and an output portion 9 prints the image of each subject for photography according to the converted image signal.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention detects information from a photographer's eyes (gaze information and pupil size information), records the information on a film, and performs image conversion based on the information when printing. The present invention relates to a line-of-sight detection camera and a printer device that create beautiful photographs.

[0002]

2. Description of the Related Art Conventionally, various kinds of information are input to a camera by using, for example, a dial or a button, and the operating environment becomes complicated as the input information increases. For example, a technique for detecting the line-of-sight direction of a photographer looking through a viewfinder and inputting information to the camera from the line-of-sight information is disclosed in Japanese Patent Laid-Open No. 63-
Many are disclosed in Japanese Patent Laid-Open No. 194237, Japanese Patent Laid-Open No. 3-87818, and the like.

In the detection method, the cornea, iris,
A large number of methods for detection using any one of the pupil, the sclera (white eye), etc. have been disclosed (JP-A-1-160537).
JP-A-2-206425, etc.).

Further, there is disclosed a method of recording visual line information and printing the visual line information position (main subject) neatly at the time of printing based on the visual line information (JP-A-4-1343).
No. 30, JP-A-5-158132). Also,
In digital image processing, gradation conversion processing and the like are generally performed.

[0005]

As described above, a technique for reproducing an optimum image based on image processing has been widely used. However, when the person in charge of creating the optimum image is different from the photographer, it is not possible to determine which part of the gradation should be changed, and the optimum processing cannot be performed.

When there are a plurality of main subjects,
Even if the line-of-sight information is left on the film to be processed optimally, an optimum image cannot be provided even if it is realized without performing gradation processing or the like. Further, the image obtained depending on the state of the eye while gazing is often different.

The present invention has been made in view of the above problems, and an object thereof is to record the line-of-sight information of a photographer on a film and restore an optimum image by post-processing based on the information on the film. To do.

[0008]

In order to achieve the above-mentioned object, a line-of-sight detection camera according to a first aspect of the present invention includes a line-of-sight detection unit for detecting the movement of a photographer's eye and detection by the line-of-sight detection unit. Main subject detection means for repeatedly detecting a main subject based on the photographed eye position information of the photographer, and information of a plurality of main subjects detected by the main subject detection means on a film or on a film cartridge. And an information recording means for recording the information.

The line-of-sight detection camera according to the second aspect includes a pupil detecting means for detecting the size of the pupil of the photographer, and the information recording means uses the size of the pupil detected by the pupil detecting means as described above. It is characterized by recording on a recording medium.

Further, the printer device according to the third aspect is a printer device for printing from a film in which information of a plurality of main subjects is recorded on a storage medium on a film or a film cartridge, wherein the plurality of storage media are used. An information reading means for reading the main subject information,
An image reading means for converting an object image exposed on the film into an image signal, and the plurality of main objects are properly printed based on the plurality of main object information read by the information reading means. Further, it is characterized by further comprising: a gradation converting means for converting the gradation of the image signal; and a printing means for printing the subject image based on the image signal converted by the gradation converting means.

[0011]

That is, in the line-of-sight detection camera according to the first aspect of the present invention, the line-of-sight detection means detects the movement of the eye of the photographer, and the main subject detection means detects the eye of the photographer detected by the line-of-sight detection means. The main subject is repeatedly detected based on the position information of 1., and the information recording means records the information of the plurality of main subjects detected by the main subject detecting means in the storage medium on the film or the film cartridge.

In the line-of-sight detection camera according to the second aspect, the pupil detection means detects the size of the pupil of the photographer,
The information recording means records the size of the pupil detected by the pupil detecting means on the recording medium.

Further, in the printer device according to the third aspect, the information reading means reads the plurality of main subject information from the storage medium, and the image reading means converts the subject image exposed on the film into an image signal. The gradation converting means converts the gradation of the image signal based on the plurality of main subject information read by the information reading means so that the plurality of main subjects are properly printed, and the printing means The subject image is printed based on the image signal converted by the gradation converting means.

[0014]

EXAMPLES Before describing the examples of the present invention,
The principle of line-of-sight detection used in the embodiments of the present invention will be described. There are various methods for detecting the line-of-sight direction. Here, as a method applicable to a camera, a corneal reflection image called a first Purkinje image and a fundus oculi which are already well known to those skilled in the art. A method of detecting using the reflection image or the edge of the rainbow collection will be briefly described. Since the structural description is already known, it is omitted here.

First, FIG. 2 shows the manner in which the reflected light from the eye of the light beam projected from substantially the optical axis is received and output. When light is projected onto an eyeball 90 (not shown) and a reflected image thereof is captured, the first Purkinje image having a high received light output has a small amount of reflected light and a position where a reflected image can be formed is different, which makes it difficult to detect.

Then, the reflected light 95b from the fundus when the light is projected onto the eyeball 90 causes the fundus image to be detected as the silhouette of the rainbow sampling edge 94 which is the peripheral image of the pupil. The reflected image 95b from the fundus is shown in FIG. 2 together with the first Purkinje image 95a, and the gaze direction is detected using these two images. Incidentally, reference numeral 91 is a rainbow harvest, 92 is a sclera (white eye), 9
3 is a pupil, and 94 is a rainbow sampling edge.

Next, FIG. 3 shows how the detected image changes due to the rotation of the eyeball. When the optical axis 98 of the eyeball 90 and the light flux projected on the eye are parallel, as shown in FIG. 3A, the center of the fundus image 95b, that is, the center of the pupil and the center of the first Purkinje image 95a coincide with each other. There is. When the eyeball 90 rotates, the optical axis 98 rotates around the rotation center 90c of the eyeball 90 as shown in FIG.

In this case, the center of the fundus image 95b can be received at different positions on the sensor pixel row that receives the reflected light from the eye. Further, the center of the first Purkinje image 95a receives light at a position relatively different from the center of the fundus image 95b. This is because the center of the curved surface on the front surface of the cornea is different from the center of rotation of the eyeball.

Therefore, the amount of rotation and the amount of shift of the eyeball 90 of the photographer looking into the viewfinder can be obtained from the displacement of the absolute positions of the two images with respect to the sensor pixel array and the relative displacement of the two images. Furthermore, it is possible to determine where the photographer is looking. In the present invention, detection is performed using at least one of the corneal reflection image 95a and the fundus reflection image 95b from the line-of-sight detection image.

Next, FIG. 4 shows the rotation and the corneal reflection 95a and the fundus reflection 95b in the case where the center of the eyeball is fixed. If the center of the eyeball is fixed, the fundus reflex 9
The rotation angle can be detected by detecting the barycentric position ix of 5b, the barycentric position px of the corneal reflection 95a, or the barycentric position including both. In the figure, px indicates the center of gravity of the corneal reflection 95a, and ix indicates the center of gravity of the fundus reflection 95b.

Further, the fundus reflection shown in FIG. 4 is generally a so-called "red eye state" in which a large amount of light is reflected from the fundus. The output of the fundus reflex 95b is further reduced when the rainbow is narrowed in a bright state or when the reflected light does not return to the light receiving system and the red eye state is not achieved. When viewed from the center of the finder, FIG. 4A shows the center at a rotation angle of 0 (reference), and FIG.
Indicates a left side with a negative rotation angle and a right side with a positive rotation angle in FIG.

Next, in FIG. 5, the eyeball shift and the corneal reflection 95
The state of a and the fundus reflection 95b will be described below. Fundus reflex 9
The approximate shift amount can be detected by detecting the barycentric position ix of 5b, the barycentric position px of the corneal reflection 95a, or the barycentric position including both.

In the figure, px indicates the position of the center of gravity of the corneal reflection 95a, and ix indicates the position of the center of gravity of the fundus reflection 95b. Further, similar to FIG. 4 described above, the fundus reflex is generally in the red-eye state, and when it is not in the red-eye condition, the fundus reflex 95
The output of b is further reduced. When viewed from the center of the finder, FIG. 5A shows the center with shift 0 (reference).
In FIG. 5B, the shift amount is negative and the left side is seen. In FIG. 5C, the shift amount is positive and the right side is seen. Generally, in one shutter sequence, that is, from the vicinity of the 1st release to the 2nd release, the eye shift does not greatly change, and the relative movement and the reference position are detected to determine the direction in which the photographer intends to see. I can know. Further, the reference position detection may be performed within the sequence.

An embodiment of the present invention based on the above principle will be described below. First, FIG. 1 shows a first embodiment of the present invention.
The configurations of the line-of-sight detection camera and the printer device according to the embodiment will be shown and described. On the side of the line-of-sight detection camera, the outputs of the line-of-sight detection unit 1 and the pupil diameter detection unit 2 are connected to the input of the subject detection unit 3, and the output of the subject detection unit 3 is connected to the input of the film recording unit 4. Then, the film recording unit 4 transmits the information to the printer device side via the film. On the other hand, in the printer device, the outputs of the image reading unit 5 and the information reading unit 6 are connected to the input of the image conversion unit 7, and the output of the image conversion unit 7 is connected to the inputs of the display unit 8 and the output unit 9. .

In such a structure, first, on the side of the line-of-sight detection camera, the line-of-sight detection unit 1 detects the direction of the line-of-sight of the photographer looking into the finder and sends the information to the subject detection unit 3. Then, the pupil diameter detection unit 2 detects the size of the pupil of the photographer and sends the information to the subject detection unit 3. The subject detection unit 3 extracts one or more main subjects from the line-of-sight information from the line-of-sight detection unit 1, and extracts information about the corresponding pupil diameter from information about the pupil size from the pupil diameter detection unit 2. The film recording unit 4 is related to the information of one or more main subjects detected by the subject detection unit 3 (information about the size of the pupil corresponding to the main subject and other information about the size of the pupil), and information about the size of the pupil. On the film (magnetic recording, optical (barcode imprint) recording).

When the film is developed and passed to the printer device in this way, the image reading unit 5 on the printer device side reads the image information with a film scanner and sends the image information to the image conversion unit 7. The information reading unit 6 reads the information recorded on the film (optical recording up to magnetic recording) (main subject information and pupil size information detected by the line of sight), and sends the main subject information to the image conversion unit 7. The image conversion unit 7 displays an image whose gradation has been converted from the image information, the main subject information, and the size information of the pupil on the display unit 8 (the image before conversion can also be displayed) and sends the image information to the output unit 9. Then, the output unit 9 outputs after the converted image is confirmed.

Hereinafter, with reference to the flowchart of FIG.
The sequence on the side of the line-of-sight detection camera will be described. When the sequence on the side of the line-of-sight detection camera is started, the line-of-sight detection unit 1 processes the line-of-sight information (line-of-sight detection and one or more main subjects) (step S1), and the pupil diameter detection unit 2 performs processing of pupil size detection. Perform (step S2). Then, the subject detection unit 3 performs a sequence regarding AF and exposure (step S
3) In synchronization with film winding (or rewinding), the film recording unit 4 records one or more main subject information detected by the line of sight and pupil size information (step S
4) Then, this sequence is finished (step S5).

Next, the sequence on the printer side will be described with reference to the flowchart of FIG. When the sequence on the printer device side is started, the information reading unit 6 reads one or more pieces of main subject information and the size of the pupil recorded on the film, that is, the film that has been developed (step S1).
1) The image reading unit 5 reads image information (step S12). Then, the display of the read image information and the indication of the main subject are performed by flashing, circles, or the like (step S13), and the image conversion unit 7 performs gradation conversion processing and confirmation on the display image (step S1).
4), the display unit 8 displays the converted image (step S15). After this confirmation, an image is output from the output unit 9 (step S16), and this sequence ends (step S).
17).

Note that the gradation conversion characteristics may be manually changed when the confirmation is NG. Further, the display unit 8
The image gradation of the output unit 9 may be adjusted. If the information recorded on the film is only the main subject information, 1
When the images of three or more main subject positions are scanned and the gradation is determined based on the density of each main subject position on the film, and only pupil size information is used, the pupil size change amount and each film block (screen The gradation may be determined according to the density of (divided into a plurality of blocks). Further, in order to secure the dynamic range of the scanner, it is advisable to change the integration time of the scanner and perform a plurality of scans, and process using the image corrected with the integration time.

Next, FIG. 8 shows the constructions of the visual axis detection camera and the printer apparatus according to the second embodiment which further embodies the first embodiment of the present invention. In the second embodiment, the one-dimensional line-of-sight detection is used, but it goes without saying that the two-dimensional detection improves the accuracy.

As shown in FIG. 8, on the side of the line-of-sight detection camera, the output of the photometry unit 23 is connected to the input of the KCPU 21 (camera side CPU), the I / F circuit 22 is the LED circuit 28, the line sensor 24, Connected to the KCPU21,
The LED circuit 28 is the I / F circuit 22, the finder optical system 2
Connected to 5. Further, the line sensor 24 is I /
The F circuit 22 and the finder optical system 25 are connected to each other. The finder optical system 25 is optically connected to the LED circuit 28 and the line sensor 24, and is also optically connected to the photographer's eyes and a shooting scene. The winding / rewinding circuit 26 and the magnetic circuit 27 are connected to the KCPU 21 and the film 29.
It is connected to the.

On the other hand, on the printer side, the magnetic reading circuit 32 is connected to the film 30 (developed), and the film scanner 33 is connected to the film 30, the PCPU 31 (CPU on the printer side) and the image memory 34. The display circuit 35 is connected to the image memory 34 and the PCPU 31. The output image memory 36 is connected to the image memory 34, the PCPU 31 and the printing unit 37. Regarding AF, known multi-AF (for example, scan AF: Japanese Patent Laid-Open No. 4-3248)
No. 07, etc.) is preferably used.

With such a structure, on the camera side,
The photometric unit 23 sends photometric information to the KCPU 22. In the line-of-sight detection block, the I / F circuit 22 controls the LED circuit 28 and the line sensor 24 and mutually communicates information with the KCPU 21, and the LED circuit 28 projects a light beam to the eye through a part of the finder optical system 25. Then, the line sensor 24 takes in the reflected light flux from the eye through the finder optical system 25. The KCPU 22 extracts information about the line-of-sight direction and the size of the pupil from the image information of the eye from the line-of-sight detection block,
One or more pieces of information about the main subject are extracted from the information and transmitted to the magnetic circuit 27 together with pupil size information, photometric information, and exposure information, and the winding / rewinding circuit 26 is controlled to the film 29 by a magnetic signal. Record.

On the other hand, on the printer device side, the magnetic reading device 32 is magnetized from the film 30 rather than the film 30.
Read the information from the camera recorded in the
Send to PU31. The film scanner 33 is the PCPU 31
Of the control signal (the optimum scanning speed is set based on the photometric information and the exposure information), the image information is read from the film 30 and sent to the image memory 34. The PCPU 31 sends the main subject information to the display circuit 35 based on the main subject information and the image, and also performs the gradation conversion (table conversion, function conversion) based on the information of the magnetic reading device to confirm the display of the processed image. , The created image is output image memory 34
Send to. The display circuit 35 performs image display and main subject display based on the image stored in the image memory 34 and main subject information from the PCPU 31. The output image memory 36 stores the newly created image. The printer device 37 outputs the created image.

Here, FIG. 9 shows the detailed construction of the visual axis detecting optical system for explanation. FIG. 9A shows an optical arrangement. The detection sensor detects reflected light from the eye using a storage line sensor. Two light sources, LED0 and LED1, are used as the light source, LED0 is arranged so that a red-eye image is generated on the detection line sensor, and LED1 is a light-projected light of LED0 so that a red-eye image is not generated on the detection line sensor. It is arranged so that the light is projected at an angle of 2 to 5 ° with respect to the axis. The light projected from the LEDs 0 and 1 is projected onto the eye through the beam splitter, and the reflected light from the eye is guided again to the line-of-sight optical system and line sensor through the beam splitter.

9 (b) and 9 (c) show LED0 and LED1.
7 shows a state of an image of an eye when is projected. For details, see FIG.
FIG. 9B shows a state in which a Purkinje image and a red-eye image are detected by the light projected from LED0, and FIG. 9C shows a state in which the Purkinje image is present but the red-eye image cannot be detected by the light projected from LED1. ing.

The positions of the Purkinje images formed by the LEDs 0 and LED1 are arranged so that they can be located at the same position in the x-axis direction (positions on the detection sensor orthogonal to the detection direction of the Purkinje images). Light source).

Next, FIG. 10 shows the signals on the line sensor shown in FIGS. 9B and 9C for explanation. The image in FIG. 10C shows the result of the difference processing of the signal in FIG. 10A and the signal in FIG. 10B, and it can be seen that only the red-eye image is extracted. Therefore, from the image shown in FIG. 10C, the position of the center of gravity of the red eye (center of the pupil) and the maximum value (Purkinje image position) can be detected, and the line-of-sight direction can be determined. It is also possible to detect the size.

The sequence on the side of the line-of-sight detecting camera in the second embodiment will be described below with reference to the flowchart of FIG. When the camera sequence starts, the sequence is initialized. That is, here, the variable i is set to 0 and the fucus point is set to the center (step S21). Then, the 1st release is determined (step S22).

Then, the 1st release switch is turned off.
If it is, the process proceeds to step S31, and if the 1st release switch is ON, the subroutine "line of sight detection 1"
Is performed to detect the line of sight in synchronization with the 1st release (step S23).

That is, as shown in the flowchart of FIG. 12, when the subroutine "visual axis detection 1" is started, a subroutine "visual axis detection" which will be described later is executed (step S
32), the line-of-sight block data is stored in the memory E (i), and the size of the pupil is stored in the memory P (i) (step S33).
The process returns to the main routine (step S34).

Subsequently, the subroutine "visual axis detection 2" is executed to perform continuous visual axis detection (step S24). That is, as shown in the flowchart of FIG. 13, when the subroutine "visual axis detection 2" is started, the variable i is incremented (step S35), and then the subroutine "visual axis detection" to be described later is executed (step S36). The line-of-sight block data is stored in the memory E (i), and the size of the pupil is stored in the memory P.
It is stored in (i) (step S37) and the process returns to the main routine (step S38).

Next, the KCPU 21 determines again the 1st release (step S25), and when the 1st release switch is OFF, moves to step S31,
If the 1st release switch is ON, the second release is determined (step S26). If the 2nd release switch is OFF, the process returns to step S24, and if the 2nd release switch is ON, multi-scan distance measurement is performed. Here, the subroutine "distance measurement" is executed in accordance with the information previously detected in the subroutine "line-of-sight detection 1" to set the distance measurement point (step S2).
7).

That is, as shown in the flowchart of FIG. 20, when the subroutine "distance measurement" is started, multi-distance measurement is performed (step S121), and the data E (0) is evaluated (step S122). If E (0) = 0 is not satisfied, an AF point is set by a known evaluation AF algorithm that does not use the line-of-sight data (step S12).
3) If E (0) = 0, the AF point is set by the information of the distance measurement block of the E (0) data (step S124). Further, the lens drive amount is calculated based on the distance measurement information of the AF point (step S125), and the process exits this sequence and returns to the main routine (step S126).

Next, the KCPU 21 performs a photographing sequence (lens drive, exposure determination, exposure) (step S2).
8) The winding is started (step S29), and the information is recorded on the film 29 in synchronization with the winding (step S3).
0), this sequence is ended (step S31).

The sequence of the subroutine "information recording" executed in step S30 is shown in FIG. That is, when the subroutine "information recording" is started, histogram processing for each block and normalization of the number of detections are performed from the eye movement detection data E (i) in the period from the 1st release to the 2nd release (step S81). , The upper 3 blocks are selected from the blocks having a predetermined value or more, and block information is output (step S82). Then, the size information of the pupil of the corresponding block is output (step S8).
3) and output other pupil size information (step S
84) and this sequence is exited (step S85).

Next, the sequence of the above-mentioned subroutine "line-of-sight detection" will be described with reference to the flowchart of FIG.
When the subroutine "line-of-sight detection" is started, initialization is performed (step S41), the sensor is reset and LED0 is projected (step S42), and the sensor signal D0 is output.
Is detected (step S43). Then, the sensor is reset, the LED 1 is projected (step S44), and the sensor signal D1 is detected (step S45). Then, signal processing for detecting the line of sight by the detection signals D0 and D1 is performed (step S46), and this sequence is exited (step S47).

In step S45, the subroutine "signal processing" is executed as shown in the flowchart of FIG.
Is started, the detected data D0 and D1 are weighted in a predetermined manner, and the data D (D = a × D0−b × D1; a,
b predetermined variable) is created (step S51). Then, the center of gravity position xd is detected from the data D (step S5
2), the MAX value position (the center of gravity position of the MAX data group when there are a plurality of MAX values) x0 and the MAX value M from the data D0
0 is detected (step S53), MAX is obtained from the data D1.
A value position (the barycentric position of the MAX data group when there are a plurality of MAX values) x1 and a MAX value M1 are detected (step S5).
4).

Subsequently, the amount of deviation between x0 and x1 is evaluated (the amount of eye movement is evaluated) (step S55). And |
If x0-x1 | <ε, the process proceeds to step S62, and if | x0-x1 | <ε, the average value Av0 is obtained from the data D0 (step S56). Next, the difference between the MAX value M0 of the data D0 and the average value Av0 is evaluated (k is a predetermined value) (step S57), and if not M0-Av0> k, the process proceeds to step S62 and M0-Av0> k.
In this case, the average value Av1 is obtained from the data D1 (step S58). Subsequently, the difference between the MAX value M1 of the data D1 and the average value Av1 is evaluated (k is a predetermined value) (step S
59), if M1-Av1> k is not satisfied, step S6.
2, the pupil diameter dp is detected from the detection signal D when M1-Av1> k (step S60).

The sequence of the "pupil diameter detection" subroutine executed in step S60 is as shown in FIG. 16 (a). That is, when the subroutine "pupil diameter detection" is started, the signal of the detection data D is binarized by a predetermined output (step S71), and the change position of the binary data in the x-axis direction as shown in FIG. 16 (b). xf and xg are detected (step S72). Then, measure the size of the pupil (d
p = xg-xf) (step S73), and this sequence is exited (step S74).

Returning to the sequence of FIG. 15 in this way, a detection block is determined from the detected red-eye barycentric position xd and Purkinje image position x0 (blocks 1 to N) (step S61). On the other hand, in step S62, the detection block is set to 0 (block number that does not exist), and this sequence is exited (step S63).

In step S55, the detection signals D0,
The determination of the eye movement in D1 is performed, and step S5
In step S59, the determination of blinking is performed. Also,
When the finder optical system has a nozzle signal source such as internal reflection, the nozzle signals detected in advance by shielding the eyepiece side of the finder and turning on the respective LEDs to detect Dd0 and Dd1 are calculated as Dd = a × Dd0− b × Dd1 (a and b are predetermined variables), and the difference between the detection signal and the noise signal Dd may be stored in the signal D.

Here, the structure of the line sensor is shown in FIG. 18 and will be described. FIG. 18A shows a block diagram of the line sensor, and BS is composed of a photoelectric conversion element. The DBS is a basic block for dark current correction by shielding the BS. Further, the shift register drives and controls each BS by a control signal from the KCPU 21. Each BS is integrated and controlled by the control signals (T) and (C) of the KCPU 21 and the monitor signal (M) output from the BS itself. The integration control performs peak value detection (when one of the BSs reaches a predetermined level, a signal for ending the integration is output) and time control. The signal (M) is recorded in the KCPU 21 and held as the signal (T) until the next integration is started, and C holds the signal. Further, the image of the eye is erased by the dark current difference in accordance with the signal of the shift register, amplified, and then A / A of the KCPU 21.
It is sent to the D circuit. Then, the integration time control is determined in the brightest state (when red eye occurs), and the integration time when red eye does not occur is determined by the control signal (T) as the integration time when red eye occurs.

Further, FIG. 18B is a diagram showing the internal circuit of the BS. It is photoelectrically converted by PD and is accumulated in the capacitor C through the gate G02 for integration control, and the gate G0
1 resets PD and C together with G02. Gate G
03 performs read amplification, and the gate G04 reads a signal according to the signal of the shift register. Then, the diode D01 detects the peak value.

The sequence of integration time control will be described below with reference to the flowchart of FIG. When the integration sequence is started, the sensor and the timer are reset (step S101), and the light emitting LED0 is determined (step S1).
02). Then, in the case of LED0 light projection, the timer is started at the same time when the integration is started (step S103).

Subsequently, the timer t is determined (step S104). If t> td (td is a predetermined limit value), the process proceeds to step S108. If t> td is not satisfied, the integration monitor (peak value is reached). The detection) signal is determined (step S105).

If the monitor signal is OFF, the process returns to step S104 to continue the integration. Monitor signal ON
In the case of, the integration is completed (step S106), the integration time tc is recorded by the KCPU 21 (step S107), the signal is read (step S113), and the process returns (step S114).

In step S102, LED0
If is not emitted, the timer is started at the same time as the integration is started (step S109). Then, the integration monitor (peak value detection: monitor determination level is step S1).
Signal is set higher than 07) (step S)
110), if the monitor signal is ON, step S108
If the monitor signal is OFF, the timer t is determined (step S111). If t> tc is not satisfied (step S111), the process returns to step S110 to continue the integration.

On the other hand, if t> tc in step S111, the integration is terminated and the process proceeds to step S113 (steps S111 and 112). Step S108
Then, if the signal is not reliable (the integration time is longer than expected: the eye does not exist, the integration end is shorter than expected: there is strong reflection other than the eye), the signal is reset, and the process proceeds to step S113. (Step S108).

If the integration time is judged and the integration time is not within the predetermined time, it may be judged that the reliability is low and the data may be invalidated. The sequence on the printer side is shown in the flowchart of FIG. 21 and will be described.

When the printer sequence starts, the PCPU
31 initializes the data (step S20
1), the magnetic reading circuit 32 reads the magnetically recorded information (step S202). Then, the film scanner 33 reads the image information, stores it in the image memory 34 (step S203), and displays the information and the image on the display circuit 35 based on the information (step S204).
Further, the PCPU 31 performs image conversion (step S
205) and display the converted image (step S20)
6). Then, the image is confirmed (step S20
8) If the result is NG, the parameter is changed and the process returns to step S204 (step S207). If the result is OK, the converted image is output (step S209), and the present sequence is exited (step S210).

Next, the sequence of the subroutine "image conversion" will be described with reference to the flowcharts of FIGS. When the image conversion is started, the average output value of the image block at the main subject position (from the larger output value to K
0, K1, K2) and the average output value (K
a) is detected (step S301). Then, the average value of the pupils at the position of the main subject (P0, P
1, P2) is detected (step S302), and the average value (Pa) of the entire pupil diameter is detected (step S30).
3).

Subsequently, the output difference of the main subject position is detected (step S304), and if K0-K2 <Kd (Kd is a predetermined value) is not satisfied (the brightness between the main subjects is large),
The process moves to step S313.

On the other hand, when K0-K2 <Kd, the change in the pupil diameter is detected (step S305), and P0-
If P2 <Pd (Pd is a predetermined value) is not satisfied, conversion for expanding the range of the dark portion and the bright portion (see (4) in FIG. 24A) is performed (step S311). In this step S305, when P0-P2 <Pd, it is compared with the average size Pa of the pupil (steps S306 and S307).

If P0-Pa≥Pv (Pv is a predetermined value), the conversion is performed so that the average output value of the block of the first main subject is slightly below the center of the conversion output range (Fig. 24 ( Refer to (3) of a) (step S31).
0). Furthermore, in the case of -Pv>P0-Pa> Pv,
The conversion is performed so that the average output value of the block of the first main subject comes to the center of the conversion output range (see (1) in FIG. 24A) (step S308). Also, −Pv ≦ P0−
In the case of Pa, conversion is performed so that the average output value of the block of the first main subject is slightly above the center of the conversion output range (see (2) in FIG. 24A) (step S30).
9).

On the other hand, when the flow shifts from step S304 to step S313, a change in pupil diameter is detected. Then, when P0-P2 <Pd (Pd is a predetermined value) is not satisfied, the block of P0 is put in the conversion output range and the range of the dark part and the bright part is widened (see (8) in FIG. 24B). Is performed (step S319). Furthermore, P
If 0-P2 <Pd, the average size Pa of the pupil is compared (steps S314 and S31).
5). Then, in the case of P0-Pa ≧ Pv (Pv is a predetermined value), the block of K0 is put in the conversion output range, and conversion is performed to widen the range of the bright portion (see (7) in FIG. 24B). (Step S318). Furthermore, −
When Pv>P0-Pa> Pv, the block of K0 is converted into the conversion output range (see (5) in FIG. 24B).
Is performed (step S316). Also, −Pv ≦ P0−P
In the case of a, the K0 block is put in the conversion output range and the conversion for expanding the range of the dark portion is performed (see (6) in FIG. 24B) (step S317). Thus, the process returns (step S312).

FIG. 24 shows a conversion curve, and the numbers in parentheses correspond to the conversion system numbers in FIGS. Then, in the conversion, the input output may be converted into a conversion output in a table. The conversion output range is determined by the output medium.

Although the embodiments of the present invention have been described above, it is needless to say that the present invention is not limited to these and various improvements and modifications can be made. For example, the line-of-sight detection method may use a method of extracting the Purkinje image position and the pupil edge from one screen using an area sensor.
A method of extracting a pupil center of gravity and a Purkinje image by time division using a D sensor may be used, and in the case of PSD, the size of the pupil is estimated by the light amount after the difference. Further, although the reliability determination is performed by blinking and moving the eyes, it is more preferable to combine the determination of contact between the camera and the face (when the eye is pressed against the camera finder) and the determination of the eye glasses. Further, the line-of-sight detection may be performed in parallel with the sequence of cameras such as AF and AE.

Also, in the extraction of the main subject, a block in which the line of sight stays for a long time (line of sight position)
You may select from. That is, as shown in the flowchart of FIG. 25, after initialization (step S401), the variable i is incremented (step S4).
02), the sequence of the above-mentioned subroutine "line-of-sight detection" is executed, and if E (i-1) = E (i) is not satisfied (step S404), if MAX1 <E (i-1), then MAX1 J (E (i-1)) is stored in (step S410), E (i-1) is stored in MAXE1 (step S411), and the process proceeds to step S412.

Then, in step S409, the MA
MAX2 <E when X1 <E (i-1) is not satisfied
If (i-1) (step S414), MA
Store j (E (i-1)) in X2 (step S41)
5), E (i-1) is stored in MAXE2 (step S416), and the process proceeds to step S412.

Further, in step S414, MA
If not X2 <E (i-1), MAX3 <E
In the case of (i-1), MAX3 has j (E (i-
1)) is stored (step S415), E (i-1) is stored in MAXE3 (step S416), and the process proceeds to step S412. In step S417, M
If not AX3 <E (i-1), step S41
Move to 2.

In step S412, J (E (i-
1)) is 0, and J (E (i)) = J (E (i)) + 1
After that (step S405), if the first release switch is OFF, the sequence ends (step S413). And 1st release switch and 2n
If the d-release switch is turned on, the process returns (step S408).

On the other hand, one or both of the line-of-sight block (line-of-sight position) at the 1st release and the block (line-of-sight position) at the 2nd release are weighted (in many cases, they are more closely related to the main subject: number of times or time). May be evaluated). Further, in the AF method as well, a passive type two-dimensional wide-field AF using an area sensor may be used, and the number of discrete multi A
F may be used. Further, the image conversion method may of course be another method as long as it uses the line-of-sight information and the pupil size information.

Further, in the subroutine "image conversion", it is preferable to make a difference in the normal conversion depending on the amount of change of the pupil as shown in FIG. 26 (FIG. 27 shows an example of conversion characteristics). That is, as shown in the flowchart of FIG. 26, main subject pupil diameter detection (P0, P1, P2, Pa)
(Step S501), if P0-P2 <Pd, the normal characteristic is converted into the changed characteristic shown by the broken line as shown in FIG. 27 if P0-P2 <Pd is not satisfied, and then the process returns. (Steps S502 to S5
04).

The conversion method may be changed according to the absolute value of the pupil. Further, in the case of only the main subject information from the line of sight, conversion may be performed by the average output of each block corresponding to the main subject information and the average output of the screen as shown in FIG. 28 (an example of conversion characteristics in FIG. 29). Shown).

That is, as shown in the flow chart of FIG. 28, main subject block output detection (K0, K1, K
2) is performed, the average output Ka is detected, and K0 to K2 are set to M0.
To M2, Ma (steps S601 to S60)
3). Then, M0 and M2 are compared with Mmax and Mmin, and the characteristics (1) to (6) shown in FIG. 29 are selected based on the comparison result (steps S605 to S6).
15).

In addition to the above, in recording information on a film, it is more preferable to add a header signal to the information to judge the content. For example (first line-of-sight block header signal) (block position)
(Pupil average diameter) (Second line-of-sight block header signal) (Block position) (Pupil average diameter) (Third line-of-sight block header signal) (Block position) (Pupil average diameter) (Pupil diameter header) (Pupil diameter 1) ( Pupil diameter 2) ... (end signal).

As described in detail above, according to the present invention, an image in which the gradation is adjusted based on the main subject extracted from the line-of-sight information and the size information of the pupil (only one may be used) in the present camera configuration. Can be obtained.

According to the above embodiment of the present invention, the following constitution can be obtained. (1) In a camera, a line-of-sight detection device that detects a movement of a photographer's eyes, a subject detection device that selects a main subject based on eye position information of a detected predetermined period, and a film recording that records the main subject information on a film A camera system comprising: a device and a printing device that performs gradation conversion based on film information. (2) In the above (1), the visual line detection device detects the visual line direction from the reflected light from the pupil and the reflected light from the cornea. (3) In the above (1), the visual line detection device detects the movement of the visual line from the reflected light from the pupil. (4) In the above (1), the line-of-sight detection device detects the line-of-sight movement from the reflected light of the cornea. (5) In (1) above, the subject detection device detects the main subject from a position where the number of times of detection is large within a predetermined period. (6) In (1) above, the subject detection device detects the main subject from a position where the continuous stay time is long within a predetermined period. (7) In (1) above, the subject detection device detects one or more main subjects in a predetermined period. (8) In the camera, the line-of-sight detection device that detects the movement of the photographer's eye, the subject detection device that selects the main subject from the detected eye position information for the predetermined period, and the size of the pupil of the photographer's eye are set. A pupil detection device for detecting, a film recording device for recording main subject information and pupil information on a film,
A camera system comprising: (9) In (8) above, the visual line detection device detects the visual line direction from the reflected light from the pupil and the reflected light from the cornea. (10) In (8) above, the visual line detection device detects the movement of the visual line from the reflected light from the pupil. (11) In (8) above, the line-of-sight detection device detects the movement of the line of sight from the reflected light of the cornea. (12) In (8) above, the subject detection device detects the main subject from a position having a large number of detections within a predetermined period. (13) In (8) above, the subject detection device detects the main subject from a position where the continuous stay time is long within a predetermined period. (14) In (8) above, the subject detection device detects one or more main subjects in a predetermined period. (15) In (8) above, the pupil detection device is characterized in that the pupil detection device uses the signal detected by the line-of-sight detection device. (16) In (8) above, the size of the pupil detected by the pupil detection device is determined by the size of the red eye. (17) In a camera, a line-of-sight detection device that detects movement of a photographer's eye, a subject detection device that selects a main subject from eye position information of a detected predetermined period, and a size of a pupil of the photographer's eye are set. A camera system comprising: a pupil detection device for detecting, a film recording device for recording pupil information on a film, and a printing device for gradation conversion based on film information. (18) In (17) above, the visual line detection device detects the visual line direction from the reflected light from the pupil and the reflected light from the cornea. (19) In (17) above, the visual line detection device detects the movement of the visual line from the reflected light from the pupil. (20) In (17) above, the line-of-sight detection device detects the movement of the line of sight from the reflected light of the cornea. (21) In (17) above, the subject detection device detects the main subject from a position having a large number of detections within a predetermined period. (22) In (17) above, the subject detection device detects the main subject from a position where the continuous stay time is long within a predetermined period. (23) In (17) above, the subject detection device detects one or more main subjects in a predetermined period. (24) In (17) above, the pupil detection device is characterized in that the pupil detection device uses the signal detected by the line-of-sight detection device. (25) In (17) above, the size of the pupil detected by the pupil detection device is determined by the size of the red eye. (26) In a printing apparatus, an image reading device that reads a video image of a film (developed), an information reading device that reads one or both of main subject information and pupil size information recorded on the film, An image conversion device that performs gradation conversion of an image based on information from an information reading device. (27) In the above (26), the information recorded by the camera includes at least one of the existence position of the main subject and the size information of the pupil of the photographer, or both. (28) In the above (26), the image conversion device is capable of confirming the converted image. (29) In the above (26), the image conversion device is characterized in that the characteristics are converted in a table. (30) In the above (26), the image conversion device is characterized in that the characteristics are converted by a predetermined function. (31) A line-of-sight detection device that detects the movement of the photographer's eyes,
A subject detection device that selects a main subject from the detected eye position information for a predetermined period, a pupil detection device that detects the size of the pupil of the photographer's eye, and a multi-distance measuring device that detects a plurality of screen distance information. , A film recording device for recording the main subject information and pupil information on the film, an image reading device for reading the image of the film (developed), and the main subject information recorded on the film Alternatively, a printing device including an information reading device that reads one or both of pupil size information and an image converting device that performs gradation conversion of an image based on the information of the information reading device is provided. A camera system characterized by the above. (32) In (31) above, the visual line detection device detects the visual line direction from the reflected light from the pupil and the reflected light from the cornea. (33) In (31) above, the visual line detection device detects the movement of the visual line from the reflected light from the pupil. (34) In (31) above, the visual line detection device detects the movement of the visual line from the reflected light of the cornea. (35) In (31) above, the subject detection device detects the main subject from a position having a large number of detections within a predetermined period. (36) In (31) above, the subject detection device detects the main subject from a position where the continuous stay time is long within a predetermined period. (37) In (31) above, the subject detection device detects one or more main subjects in a predetermined period. (38) In (31) above, the pupil detection device is characterized in that the detection is performed using the signal detected by the line-of-sight detection device. (39) In (31) above, the size of the pupil detected by the pupil detection device is determined by the size of the red eye. (40) In (31), the information recorded by the camera includes at least one of the existence position of the main subject and the size information of the pupil of the photographer, or both. (41) In the above (31), the image conversion device is capable of confirming the converted image. (42) In the above (31), the image conversion device is characterized in that the characteristics are converted in a table. (43) In (31) above, the image conversion device is characterized in that the characteristics are converted by a predetermined function.

[0080]

According to the present invention, it is possible to provide a line-of-sight detection camera and a printing apparatus which record the line-of-sight information of a photographer on a film and restore an optimum image by post-processing based on the information of the film. it can.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a line-of-sight detection camera and a printer device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a light reception output of reflected light from an eye of a light beam projected from a substantially optical axis.

FIG. 3 is a diagram showing how a detected image changes due to rotation of an eyeball.

FIG. 4 is a diagram showing rotation and corneal reflection 95a and fundus reflection 95b when the center of the eyeball is fixed.

FIG. 5: Eye shift and corneal reflex 95a and fundus reflex 95
It is a figure which shows the mode of b.

FIG. 6 is a flowchart showing an operation on the side of the line-of-sight detection camera of the first embodiment.

FIG. 7 is a flowchart showing an operation on the printer side of the first embodiment.

FIG. 8 is a diagram showing configurations of a line-of-sight detection camera and a printer device according to a second embodiment, which is a further implementation of the first embodiment.

FIG. 9 is a diagram showing a detailed configuration of a line-of-sight detection optical system.

FIG. 10 is a diagram showing signals on the line sensor in FIGS. 9B and 9C.

FIG. 11 is a flowchart showing an operation on the side of the line-of-sight detection camera of the second embodiment.

FIG. 12 is a flowchart showing a sequence of a subroutine “visual axis detection 1”.

FIG. 13 is a flowchart showing a sequence of a subroutine “visual axis detection 2”.

FIG. 14 is a flowchart showing a sequence of a subroutine “visual axis detection”.

FIG. 15 is a flowchart showing a sequence of a subroutine “signal processing”.

FIG. 16 is a flowchart showing a sequence of a subroutine “pupil diameter detection”.

FIG. 17 is a flowchart showing the sequence of a subroutine “information recording”.

FIG. 18 is a diagram showing a configuration of a line sensor.

FIG. 19 is a flowchart showing a sequence of integration time control.

FIG. 20 is a flowchart showing a sequence of a subroutine “distance measurement”.

FIG. 21 is a flowchart showing an operation on the printer side of the second embodiment.

FIG. 22 is a flowchart showing a sequence of a subroutine “image conversion”.

FIG. 23 is a flowchart showing a sequence of a subroutine “image conversion”.

FIG. 24 is a diagram showing a conversion curve according to the second embodiment.

FIG. 25 is a flowchart showing a sequence of a subroutine “visual axis detection” according to an improved example.

FIG. 26 is a flowchart showing a sequence of a subroutine “image conversion” according to an improved example.

FIG. 27 is a diagram showing a state of conversion characteristics corresponding to FIG. 26.

FIG. 28 is a flowchart showing a sequence of a subroutine “image conversion” according to an improved example.

FIG. 29 is a diagram showing a state of conversion characteristics corresponding to FIG. 28.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Gaze detection part, 2 ... Pupil diameter detection part, 3 ... Subject detection part, 4 ... Film recording part, 5 ... Image reading part, 6 ... Information reading part, 7 ... Image conversion part, 8 ... Display part, 9 ... Output section.

Claims (3)

[Claims] 1. A line-of-sight detecting means for detecting movement of a photographer's eyes, and a main subject detecting means for repeatedly detecting a main subject based on position information of the photographer's eyes detected by the line-of-sight detecting means, A line-of-sight detection camera, comprising: information recording means for recording information of a plurality of main subjects detected by the main subject detection means on a storage medium on a film or a film cartridge. 2. A pupil detecting means for detecting the size of the pupil of the photographer is further provided, and the information recording means records the size of the pupil detected by the pupil detecting means on the recording medium. The line-of-sight detection camera according to claim 1. 3. A printer device for printing from a film in which information of a plurality of main subjects is recorded on a storage medium on a film or a film cartridge, and information reading means for reading out the plurality of main subject information from the storage medium. Image reading means for converting the exposed subject image on the film into an image signal, and the plurality of main subjects are properly printed based on the plurality of main subject information read by the information reading means. As described above, it is provided with a gradation converting means for converting the gradation of the image signal, and a printing means for printing the subject image based on the image signal converted by the gradation converting means. Printer device.