JP3036285B2 - Red eye position detector - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a red-eye position detecting device for correcting a red-eye of a person image of a subject which may occur during flash photography.

[0002]

2. Description of the Related Art When flash photography is performed using a person as a subject, a part of the flash light is reflected by blood vessels in an eyeball, and the like.
A so-called red-eye phenomenon may occur, which is returned to the camera and the center of the iris becomes red. In particular, the red-eye effect is more likely to occur due to the approach of the optical axis of the photographing lens and the optical axis of the flash with the recent miniaturization of cameras. Therefore, in order to prevent the occurrence of such red-eye, a camera with a red-eye prevention function that performs pre-emission immediately before the main emission of the flash and shoots in a state where the pupil is narrowed is commercially available. Further, when printing on photographic paper, red-eye is also corrected.

[0003]

However, even with the above-mentioned camera with a red-eye prevention function, the occurrence of red-eye has not always been able to be prevented under the individual difference of the subject person and other photographing conditions. In addition, red-eye correction at the time of printing on photographic paper is not general in view of the need for frequent and skillful work in the correction work.

On the other hand, a pupil position is extracted from an image picked up by an image pickup means using a pattern recognition technique or the like.
Alternatively, it is conceivable to extract a red portion in the image and perform color conversion on this portion. However, if the red eye position is to be accurately detected, the image processing technique becomes extremely complicated and the device itself becomes expensive. . On the other hand, in the ordinary image processing technique, there is a risk that the red portion other than the red-eye position may be uniformly or erroneously converted into a color, which may rather provide an unnatural photograph.

[0005] The present invention has been made in view of the above,
It is an object of the present invention to provide a red-eye position detecting device that picks up an image by an image pickup unit, extracts a plurality of color component images specific to the face from the image, and detects the red-eye position of the subject person.

[0006]

According to the present invention, there is provided a red-eye position detecting apparatus, comprising: an image pickup means for extracting at least one of a low saturation area and a low illuminance area from image data obtained by the image pickup means; Extracting means; second extracting means for extracting a flesh color area from the image data; third extracting means for extracting a red area from the image data;
And detection means for detecting a red-eye position in the image using a signal from the third extraction means.

[0007]

According to the present invention, at least one of the low saturation area and the low illuminance area and the skin color area are extracted from the image data obtained by the imaging means, and for example, the logical product is obtained by using the extracted signals. , An area including the eyes of the person is extracted. Further, using the extracted red part signal,
When a red eye is generated from the area including the eye, the position of the red eye is detected.

The red-eye position data thus obtained is corrected such that the red at the red-eye position becomes a different color, for example, a normal black eye of a person, when the captured image data is read out to a monitor or the like. Used as

[0009]

FIG. 1 is an overall block diagram showing a first embodiment of a red-eye correction apparatus to which the present invention is applied. In the figure,
Reference numeral 1 denotes an A / D converter that converts R, G, and B color image signals obtained by color imaging of a developed color film by the imaging unit 20 into, for example, 8-bit digital signals for each pixel. The image memory 2 includes memory units 201, 202, and 203 for storing, for each color, R, G, and B image signals input in synchronization with the A / D converter 1, respectively. It has a storage capacity corresponding to one frame.

As shown in FIG. 2, the image pickup section 20 is provided with a winding means 22 for pulling out the developed color film F from the film cartridge 21 and winding the same, and between the film cartridge 21 and the winding means 22. A light source 23 and a mirror 24 are provided so as to be opposed to each other so as to sandwich the film F to be drawn out.
It is constituted by an image pickup means 26 such as a CCD provided with each filter of B color, and can be fed out one frame at a time by the manual instruction or one frame at a time by the winding means 22 automatically. The unwinding by the winding means 22 one frame at a time can be performed via optical means (not shown) for detecting the perforation of the film. A recording medium such as a magnetic tape or the like is provided at an appropriate position of the film cartridge 21 or at a lower end position of the film corresponding to each frame of the film F. In the case where other required photographing information can be recorded, a magnetic head 27 is attached so that the photographing magnification β and the like can be read as needed.

Returning to FIG. 1, the L, u, v chromaticity matrix circuit 3 reads out R, G,
The B signal is converted into an L, u, v color system (based on the International Commission on Illumination CIE: JIS8730). LUT4
LTU6 incorporates a plurality of tables, respectively, and performs conversion processing and the like, which will be described later, on the input signal using the selected table and outputs the result. Table 1 shows LUT4 to LUT
4 shows a table built in TU6.

[0012]

[Table 1]

The microcomputer 7 sends R / W control and address control signals to the image memory 2 and a binary data memory 13 to be described later, selection signals for each table of the LUTs 4 to 6, and switching signals for switches SW1 to SW4 to be described later. It is output as needed. The microcomputer 7 controls a required circuit section so as to repeatedly read out the image signal from the image memory 2 at a predetermined cycle, thereby providing a still image to a monitor or the like. Further, the microcomputer 7 can read and input a photographing magnification β for each frame of the filmed film from the magnetic head 27, and uses the photographing magnification β to change the filter characteristics of the low-pass filters LPF8 to LPF8 to be described later. Things. When the imaging unit 20 is of a type that cannot automatically read the photographing magnification β, it may be one that allows manual input.

The LPFs 8 to 10 provide a so-called filter effect for dulling an input signal. For example, the LPFs 8 to 10 filter a signal input from the LUT 4 via a low illuminance detection table to detect low illuminance in an image. Try to increase the range. INV11 inverts the input level and outputs the result. The binary data memory 13 stores the logical product output from the AND circuit 12. The binary data memory 13 has an address corresponding to the memory section of each color of the image memory 2. Further, the output of the binary data memory 13 is transmitted through the switches SW 4 and LPF 10 based on a control signal from
The input can be re-input to one input terminal of the D circuit 12.

The switches SW1 to SW4 are switches for switching signal paths, and are controlled by a switching signal from the microcomputer 7. Switch SW1 and switch SW
2 are connected in series, and the output of LUT 6 is connected to LPF 8
And the output of LUT6 is directly connected to A via INV11.
It switches between the case where it is guided to the ND circuit 12 (the state shown). The switch SW3 switches between a case where the output of the LUT 5 is guided to the LPF 9 and a case where the output of the LUT 5 is directly guided to the AND circuit 12 (the state illustrated). Switch SW4 is L
Switching between the case where the output of the UT 4 is guided to the LPF 10 (the state shown) and the case where the output from the binary data memory 13 is guided to the LPF 10 is performed. At this time, a switch may be added to the output side of the switch SW4, and the output from the binary data memory 13 may be directly led to one input terminal of the AND circuit 12 without passing through the LPF 10.

The matrix circuit 14 converts signals input from the LUTs 4 to 6 into Y, RY, and BY signals. The encoder 15 encodes each signal from the matrix circuit 14 into a Y signal and a C signal or into an NTSC signal. The D / A converter 16 converts an output signal of the encoder 15 into an analog signal and outputs the analog signal to a monitor (not shown).

Next, the detection of the red-eye position and the color correction thereof will be described with reference to FIGS. FIG. 3 is a diagram for explaining a red-eye position, FIGS. 4A to 4J are diagrams for explaining image processing contents for each detection procedure of the red-eye position, and FIGS. FIG. 9 is a diagram illustrating an algorithm of a correction process.

Now, assuming that the film image is a color human image shown in FIG. 4A, this image is picked up by the image pick-up section 20 and taken in as R, G, and B image signals, and A / A
The digital signal is converted into an 8-bit digital signal by the D converter 1 and stored in the image memory 2 (# 2 to # 6).
Subsequently, based on the address control signal from the microcomputer 7, the R, G, and B signals are read out synchronously,
The chromaticity matrix circuit 3 converts the signals into L, u, v signals (# 8). Then, processing A
(# 10) is executed.

FIG. 6 is a diagram showing the algorithm of the process A in detail, and includes # 30 to # 34, # 36 to # 40, and # 42.
To # 46 are synchronously processed as a first routine (first reading cycle). That is, in # 30 and # 32, the LUT
The low illuminance detection table is selected as the internal table of No. 4, whereby the low illuminance portion of the L signal becomes “1”,
The other parts are binarized as “0” (FIG. 4
(b)). In # 36 and # 38, the low chroma detection table is selected as the internal table of the LUT 5, whereby the low chroma portion is set to "1" using the u and v signals, and the other portions are set to "0". "(See FIG. 4 (c)). Further, in # 42 and # 44, the flesh color detection table is selected as the internal table of the LUT 6, whereby the flesh color portion is set to "1" using the u and v signals, and the other portions are set to "0" to be binary. (FIG. 4D).

These binarized signals are respectively guided to a matrix circuit 14 and LPF
8, 9, 10 (# 34, # 40, # 46), low illuminance area (FIG. 4 (e)), low chroma area (FIG. 4 (f)) and skin color area (FIG. 4 (g)) ). Here, the degree of expansion of the low illuminance area is determined based on the photographing magnification β corresponding to each frame. For example, the greater the imaging magnification β, the greater the degree of signal blunting. Also, # 34, #
The process 40 focuses on the fact that a low-illuminance and low-saturation black-eye portion remains around the red eye as shown in FIG. 3, and the extraction range is expanded to encompass this black-eye portion. This is to ensure that the red-eye occurrence position is within the detection range. The process of # 40 utilizes the fact that red eyes occur in a portion surrounded by flesh color, so that the entire face or the like is included in the detection range.

2 output synchronously from LPFs 8 to 10
The value signal is ANDed by the AND circuit 12 (FIG. 4).
(h)) Stored in the binary data memory 13 (# 48 to # 48)
# 50).

Subsequently, # 52, # 54, # 56 to # 60
And a second routine (second reading cycle) of # 62 and # 64 is executed. That is, in # 52, the red detection table is selected as the internal table of the LUT 5, whereby
Using the u and v signals, the red portion is binarized as "1" and the other portions are binned as "0"(# 54, (FIG. 4
(i))). At this time, the switch SW3 is switched, and the binary signal from the LUT 5 is directly guided to the AND circuit 12 without passing through the LPF 9. Furthermore, # 56
Then, a flesh color detection table is selected as an internal table of the LUT 6, whereby the flesh color portion is binarized to "1" and the other portions to "0" using the u and v signals (# 58). . This binary signal is inverted at INV11 by switching the switch SW1 (# 60), and furthermore, LPF8
, And directly to the AND circuit 12. The processes of # 52, # 54, and # 56 to # 60 are performed in the region detected as the skin color when the red region and the skin color region sometimes overlap due to various conditions such as individual differences and shooting light sources. On the other hand, this is a preparation process for prohibiting color correction except for the red-eye position.

The signals from the LUT 5 and INV 11 are ANDed by the AND circuit 12 together with the binary data (FIG. 4 (h)) synchronously read from the binary data memory 13 by switching the switch SW4. Being
It is stored again in the binary data memory 13 (# 62, # 6
4). As a result, in the binary data memory 13, FIG.
As shown in (1), only the red-eye position is captured as data "1". Next (the third reading cycle), the process returns to FIG. 5, and the process B (# 12) is executed. At this time, the binary signal passed through the switch SW4 may be led to the AND circuit 12 without passing through the LPF 10 as described above.

FIG. 7 is a diagram showing the algorithm of the process B in detail. In the process B, the microcomputer 7 sequentially reads out the binary data from the binary data memory 13 in synchronization with the reading of the image signal from each color memory section of the image memory 2 (# 70). Then, the binary data memory 1
It is determined whether the output value from # 3 is "1" or "0", that is, whether it is the red-eye position or not (# 72). The microcomputer 7 sets the LUT according to the binary data "1" and "0".
The tables built in 4 to 6 are changed. That is,
If the binary data is "0", all of the LUTs 4 to 6 are switched to the through table, and the 8-bit input signal is output as it is without conversion (# 74).
On the other hand, if the binary data is “1”, the process C is executed (# 76).

FIG. 8 shows the details of the algorithm of the processing C. The LUT 4 is switched to a table for lowering the low level (# 80, # 82), and the LUTs 5 and 6 are switched to the red suppression table, and the red eye position is reduced. The red color is subjected to the suppression processing, and is corrected, for example, to black eyes (# 84, # 86, # 8)
8, # 90).

The respective signals (# 82, # 86, # 90) output from the LUTs 4 to 6 are converted by the matrix circuit 15 into respective signals of Y, RY, and BY. The encoder 16 converts the signal into a Y signal and a C signal, or NT
It is coded into an SC signal, converted into an analog signal by a D / A converter, and converted into a Y or C signal or NT.
It is output as an SC signal (# 14 to # 20). The Y, C signal or NTSC signal is guided to a monitor or the like (not shown) and reproduced as a still image. After starting playback,
Once the red-eye position is detected, it is not necessary to continuously perform such detection processing for each readout cycle.
The signal lines from 4 to 6 to the AND circuit 12 side or the control between the AND circuit 12 and the binary data memory 13 may be controlled. These points are the same in the embodiments described later.

In this embodiment, an image which is not red-eye corrected during the first two cycles from the image memory 2 is guided to the monitor, and an image which is red-eye corrected after the third cycle is guided to the monitor. However, the time during this period is short, and does not give any discomfort to the observer. Alternatively, the output of the image signal may be cut off by any of the matrix circuits 14 to the D / A converter 16 until the correction data is obtained in the binary data memory 13.

FIG. 9 is an overall block diagram showing a second embodiment of the red-eye correction device. In the figure, components denoted by the same reference numerals as those in FIG. 1 perform the same functions. The imaging unit is omitted from the figure.

In the configuration of the second embodiment, switches SW10 and SW11 and multipliers 17 and 18 are provided between the image memory 2 and the matrix circuit 3. Switch S
W10 switches between the R color memory unit 201 and the multiplying unit 17 to be connected to the chromaticity matrix circuit 3;
W11 switches between the G color memory unit 202 and the multiplying unit 18 to connect to the chromaticity matrix circuit 3.

The multiplication unit 17 is interposed between the B color memory unit 203 of the image memory 2 and the switch SW10, and multiplies the output from the B color memory unit 203 by a coefficient k1.
The multiplication unit 18 is interposed between the B color memory unit 203 of the image memory 2 and the switch SW11, and multiplies the output from the B color memory unit 203 by a coefficient k2. Therefore, when the switches SW10 and SW11 are switched to the multiplication units 17 and 18, the chromaticity matrix circuit 3 stores (R ',
G ′, B ′) = (k1 · B, k2 · B, B). The microcomputer 7 controls these switches SW10,
SW11 is appropriately switched as described later. Table 2 shows L
4 shows a table built in the UTs 41, 51, and 61.

[0031]

[Table 2]

The red-eye correction processing in the second embodiment is as follows.
In FIG. 5 of the first embodiment, the content of the process B (# 12) is different. FIG. 10 is a diagram showing the algorithm of this processing in detail. That is, the microcomputer 7 sequentially reads binary data from the binary data memory 13 and, in synchronization with this, reads image signals of each color from the image memory 2 (# 100). Then, it is determined whether the binary data read from the binary data memory 13 is “1” or “0”. If the binary data is "0", the LUTs 41, 51,
Each of the built-in tables 61 is switched to a through table, and the 8-bit image signal from the chromaticity matrix circuit 3 is not converted and the matrix circuit 1 is not converted.
5 (# 104).

On the other hand, when the binary data is "0", the LU
The built-in tables of T41, 51, and 61 are the through tables as they are, but the switches SW10 and SW11 are switched to the multipliers 17 and 18 to perform the following signal processing. That is, (R ′,
G ′, B ′) = (k1 · B, k2 · B, B) and an 8-bit signal converted is input.

In this way, compared to the first embodiment, the level conversion table as the table in the LUT 41 and the red extreme pressure table as the tables in the LUTs 51 and 61 become unnecessary. Reduction can be achieved.

Next, FIG. 11 is an overall block diagram showing a third embodiment of the red-eye correction device, and FIG. 12 is a diagram showing an algorithm of the red-eye position detection processing. In the figure, components denoted by the same reference numerals as those in FIG. 9 perform the same functions. The imaging unit is omitted from the figure.

The LUT 52 has a through table and a red detection table, and the LUT 62 has a through table and a skin color detection table. Note that the L signal from the chromaticity matrix circuit 3 is not used when detecting the red position, and is only input directly to the matrix circuit 14 during red-eye correction, so that the LUT 41 shown in FIG. 9 is unnecessary.

The contour extracting section 31 extracts a region where a red portion exists as a block from binary data sequentially output from each address of the binary data memory 13 and extracts a contour of the region. The shape identification unit 32 uses a pattern recognition technique to determine whether or not the outline of the extracted red portion is circular, and identifies the circular shape as a red-eye position. The LPF 81 has a filter characteristic that can be changed according to the photographing magnification β from the microcomputer 71.
It is assumed that the larger the imaging magnification β, the larger the face portion of the person is, and the higher the filtering effect. A voltage corresponding to level "1" is applied to one input terminal of the switches SW5 and SW6.

Next, the red-eye position detection processing will be described with reference to FIG.
A description will be given using the algorithm of FIG. Now, it is assumed that the film image is the color human image shown in FIG. 4A, and that this image is captured as image signals of R, G, and B colors and stored in the image memory 2. Then, based on the address control signal from the microcomputer 7, the above R,
The G and B signals are read out in synchronism, converted into L, u, and v signals by the chromaticity matrix circuit 3, and are converted to # 110 and # 112.
To # 116 are processed synchronously. That is, LUT5
2, the red detection table is selected as the internal table,
As a result, using the u and v signals, the red portion is binarized as "1" and the other portions are set as "0"(# 110, # 110).
FIG. 4 (i)). On the other hand, a flesh color detection table is selected as an internal table of the LUT 62, whereby the flesh color portion is binarized to “1” and the other portions to “0” using the u and v signals (# 112, FIG. 4 (d)).

Then, the signal from the LUT 52 is supplied to the AND circuit 12 via the switch SW5, and the signal from the LUT 62 is subjected to the area expansion of the flesh color portion via the switches SW1, SW2 and the LPF 81 (# 114, FIG. g)), A
It is led to the ND circuit 12. Here, the degree of area enlargement of the skin color portion is determined based on the photographing magnification β corresponding to each frame. For example, the greater the imaging magnification β, the greater the degree of signal blunting. At this time, the switch SW6 has been switched to the level "1". Therefore, the logical product of the red portion and the enlarged flesh color area is obtained by the AND circuit 12 and is taken into the binary data memory 13.

Next, in the next cycle, the switches SW1, SW2, SW5, and SW6 are switched based on the switching signal from the microcomputer 7, and the binary signals are similarly read from the LUTs 52 and 62 based on the address control signal. The signal output from the LUT 62 is inverted by the INV 11, and is directly guided to the AND circuit 12 via the switches SW 1 and SW 2 without passing through the LPF 81. on the other hand,
The switch SW5 is switched to the level "1", and the switch SW6 is connected to the binary data memory 13, so that the AND circuit 12 executes the processing of # 116 and # 118. In this way, the red part existing near the flesh color area is extracted.

Subsequently, the red signal is sequentially read out from the binary data memory 13, and the outline of each red signal is extracted by image processing in the outline extraction unit 31 (# 12).
0). Then, each of the obtained contour signals is guided to the shape discriminating section 32 at the next stage. If the contour is circular, that is, circular, it is determined to be a red-eye position. Otherwise, another red object (for example, FIG. i), etc.) (# 122). The contour data determined as the red-eye position is taken into the microcomputer 71.

In the next cycle, when reading out the signal from the image memory 2, every time the address where the red-eye position exists is specified, the switches SW10 and SW11 are switched, and the (R ', G ', B') = (k1 · B, k
2 · B, B). In this case,
As in the second embodiment, a through table is selected for each of the LUTs 52 and 62.

Further, in order to perform the red-eye discrimination in the shape discriminating section 32 more accurately, the distance near the circular signal obtained by the shape discriminating section 32, that is, the distance converted to the interocular distance on the screen, is equivalent to the distance. It is also possible to determine whether or not the circular signal is a red-eye part based on whether or not the circular signal is present (# 124). Alternatively, the shape identification unit 32 calculates the area of the red portion extracted by the contour extraction unit 31, for example, a value converted to a value obtained by adding the number of red addresses, to the photographing magnification β.
The red-eye portion can be identified based on whether or not the size is expected from # (# 126). In addition, the accuracy may be further improved by taking a logical product of the red-eye detection data obtained in the binary data memory 13 in the first and second embodiments.

Next, FIG. 13 is an overall block diagram showing a fourth embodiment of the red-eye correction device, and FIG. 14 is a diagram showing an algorithm of the red-eye position detection processing. Note that, in the drawings, those denoted by the same reference numerals as those in FIGS. 9 and 11 perform the same functions.

In the figure, a labeling section 33 extracts each red area where a red signal exists as one lump, assigns a serial number to each of these red sections, and estimates the red area from the photographing magnification β. This is to extract a pair existing at the determined binocular distance.

Next, the red-eye position detection processing will be described with reference to FIG. 14. First, based on the address control signal 3 from the microcomputer 72, the R, G, and B signals are read out in synchronization with each other, and the chromaticity matrix is read. The signals are converted into L, u, v signals by the circuit, and the processing of # 130 to # 136 is executed.
That is, the red detection table is selected as the internal table of the LUT 52, whereby the red portion is set to “1” and the other portions are set to “0” using the u and v signals, thereby being binarized (# 130). .

The binary signal is subjected to image processing in the labeling section 33, and a serial number is assigned to each of the obtained red areas (# 132), and the coordinate values of each of the labeled red areas,
That is, the address data on the image memory 2 is
It is led to 2. The microcomputer 72 obtains and temporarily stores the obtained center coordinate value of the red area, that is, the center address,
On the other hand, the binocular distance predicted from the photographing magnification β is converted into a distance on an address. Then, two of the obtained center addresses are compared with the address distance corresponding to the binocular distance for all the combinations for all combinations, and the two red areas when they match each other are determined as red-eye parts (# 136). . Then, the microcomputer 72 stores, for example, “1” for correction in the address of the area determined as the red-eye portion. It should be noted that no correction is necessary for the other areas, and as a result, "0"
It is said. The processing at the time of red-eye correction is the same as that of the third embodiment, and a description thereof will be omitted.

In FIG. 13, a contour extracting unit 31 and a shape discriminating unit 32 surrounded by parentheses show a modified example of the red-eye position detection in this embodiment, in which the contour of each labeled red area is contoured. The extraction unit 31 extracts each of them, and the shape identification unit 32 determines that each of the outlines is circular if the outline is circular, and otherwise determines that it is another red object (# 138). This is an improvement in detection accuracy. In addition, the detection accuracy may be further improved by taking the logical product of the red-eye detection data obtained in the binary data memory 13 in the first and second embodiments.

FIG. 15 shows the fifth and sixth red-eye correction devices.
FIG. 16 is an overall block diagram showing an embodiment. FIG. 16 shows an algorithm of a red-eye position detection process in the fifth embodiment.
FIG. 7 is a diagram illustrating an algorithm of a red-eye position detection process in the sixth embodiment. The fifth embodiment is a case where the switch SW7 is in the illustrated state, and the sixth embodiment is a case where the switch SW7 is directly connected to the microcomputer 73.
Note that, in the drawings, those denoted by the same reference numerals as those in FIGS. 9 and 11 perform the same functions.

In the figure, a contour extracting section 34 extracts a contour of a region including a flesh color signal. Shape identification unit 35
Previously stores the human face shape as a reference shape,
The face is identified by determining, using a pattern recognition technique, which one or a plurality of contours obtained by the contour extraction unit 34 matches the reference shape. The painting processing unit 36 paints the inside of the closed area of the face shape identified by the shape identification unit 35, that is, writes “1” into the entire area.

Next, the red-eye position detecting process in the case of the fifth embodiment will be described with reference to the algorithm shown in FIG. First, based on an address control signal from the microcomputer 7, the R, G, and B signals are read out in synchronization and converted into L, u, and v signals by the chromaticity matrix circuit 3, and are converted into # 150 and # 150. The processing of 152 to # 158 is executed in synchronization. That is, the red detection table is selected as the internal table of the LUT 52, whereby u,
Using the v signal, the red part is binarized as "1" and the other parts are set as "0"(# 150). This binary signal is directly guided to one input terminal of the AND circuit 121. On the other hand, a flesh color detection table is selected as an internal table of the LUT 62, whereby the flesh color portion is set to "1" using the u and v signals, and the other portions are binarized as "0".

The binary signal is input to the contour extraction unit 34, and the contour is extracted for each of the regions where the skin color portion exists as a region (# 152). Further, each of the extracted contours is discriminated by the shape discriminating section as to whether or not it is a reference for the shape of the face of a person, for example, an egg shape (# 156). Then, when the contour is identified as an oval, uniform filling processing is performed on the inside of the contour (# 158). Next, the face area signal which has been subjected to the filling processing is subjected to LPF 82 to expand the area in accordance with the photographing magnification β, and then guided to the other input end of the AND circuit 121, where the logical product of the red part and the red part is obtained. (# 160). By this logical product, a red portion in the face is detected. Next, the outline of the obtained red portion is extracted (# 162), and it is further determined whether or not the extracted outline is a circle, that is, a circle (# 164).
If it is circular, it is determined that the position is the red-eye position, and the signal in the area is taken into the microcomputer. Then, the obtained red-eye position signal is guided to the microcomputer 73 and subjected to red-eye correction processing in the same manner as in the second embodiment.

Further, in order to identify the red-eye portion more accurately, it is determined whether or not there is an equivalent circular signal near the obtained circular signal in the shape discriminating unit 32. Can also be determined (# 16
6). Alternatively, the shape identifying unit 32 can identify the red-eye portion based on whether or not the area of the red portion extracted by the contour extracting unit 31 is the size expected from the imaging magnification β (# 168). In addition, the accuracy may be further improved by taking a logical product of the red-eye detection data obtained in the binary data memory 13 in the first and second embodiments.

Next, red eye position detection processing in the case of the sixth embodiment will be described with reference to the algorithm of FIG. In the sixth embodiment, the shape identification unit 35 shown in FIG. 15 identifies the shape of the eye instead of the shape of the human face, and stores the reference shape of the human eye in advance. Using a pattern recognition technique to determine which of one or a plurality of contours obtained by the contour extracting unit 34 matches the reference shape. First, as in the fifth embodiment,
The red detection table is selected as the internal table of the LUT 52, whereby the red portion is set to “1” and the other portions are set to “0” using the u and v signals to be binarized (# 180). This binary signal is supplied to the AND circuit 12 as it is.
1 to one input terminal. On the other hand, a skin color detection table is selected as an internal table of the LUT 62, and
Using the u and v signals, the skin color part is binarized as "1" and the other parts are set as "0".

The binary signal is input to the contour extraction unit 34, and the boundaries between the skin color part and other color parts are respectively extracted as contours (# 182). For example, since the face and the eyes in the face have different colors, a contour is formed on the outer periphery of the eyes as a result. In this manner, each of the extracted contours is determined by the shape identification unit 35 as to whether or not it has a standard elliptical shape, for example, a horizontal ellipse, which is a reference of the eye shape of a person (# 1).
84). Then, when the contour is identified as an eye shape (# 186), a uniform filling process is performed on a closed region surrounded by the contour (# 188). Then
The eye region signal that has been subjected to the filling process is subjected to ANDF after the region is enlarged through the LPF 82 in accordance with the photographing magnification β.
The signal is led to the other input terminal of the circuit 121, where it is logically ANDed with the red part (# 190). By this logical product, a red part in the eye is detected. Then, the obtained red-eye position signal is guided to the microcomputer 73 and subjected to red-eye correction processing in the same manner as in the second embodiment.

In order to more accurately perform red-eye identification, the red-eye portion can be identified by determining whether there is a signal having an equivalent shape near the obtained eye position signal (# 192). Alternatively, the shape identification unit 35 can identify the red-eye portion based on whether or not the area of the eye shape extracted by the contour extraction unit 34 is a size expected from the photographing magnification β (# 194). Also, the first embodiment,
In the second embodiment, a logical AND with the red-eye detection data obtained in the binary data memory 13 may be taken to further improve the accuracy. Further, the fifth embodiment and the sixth embodiment may be combined to perform more accurate red-eye position detection. Further, if the third embodiment is combined with this,
More accurate red-eye position detection can be expected.

Next, FIG. 18 is an overall block diagram showing a seventh embodiment of the red-eye correction device. In the drawing, those denoted by the same reference numerals as in FIG. 9 perform the same functions. FIG. 19 is a diagram showing an algorithm of the red-eye position detection processing.

In the figure, the microcomputer 74 is connected to the monitor 19
The upper pressed position coordinates (x, y) are input. The coordinates (x, y) correspond to the addresses of the image memory 2. In this method of specifying the position on the monitor, a transparent touch panel 191 is superposed on the screen of the monitor 19, and the touch panel 191 is provided.
Is performed, or by operating a display marker on the screen that can be instructed to move by an operation from the operation unit.

FIG. 20 is a diagram for explaining the conversion from the signal (L, u, v) to the luminance system (L) and the polar coordinate system (r, φ). FIG. FIG. 20B shows a case where the luminance signal L is extracted.

The red-eye correction table 37 converts a signal (L, u, v) at a designated position input from the image memory 2 through the switches SW10 and SW11 in the illustrated state into a luminance system (L) and a polar coordinate system (r, φ). ) And FIG.
(A) As shown in (b), a luminance signal (L ± d) and a chrominance signal (r ± α, φ ± β) in which an allowable range is added to the conversion value.
, And further inversely transforms the color signal to obtain (u ₁ ≦ u ≦ u ₂ , v ₁ ≦ v ≦ v ₂ ). The microcomputer 74 compares this value with the switches SW8 and SW9.
Is written to the RAM in the LUTs 43 and 53. The LUT 43 outputs “1” when the signal (L) sequentially read from the image memory 2 falls within the range of (L ± d), and outputs “0” otherwise. The LUT 53 outputs “1” when the signals (u, v) sequentially read from the image memory 2 are included in the range of (u ₁ ≦ u ≦ u ₂ , v ₁ ≦ v ≦ v ₂ ), and otherwise outputs In this case, "0" is output. At this time, the binary outputs from the LUTs 43 and 53 are guided to the AND circuit 121 through the switches SW8 and SW9 in response to a switching signal from the microcomputer 74.

Next, the red-eye position detection processing will be described with reference to FIG. 19. First, when a position (x, y) is specified for the red-eye position from the reproduced image on the monitor screen (# 2)
00), the address (x, y) is read from the microcomputer 74 and supplied to the image memory 2. Then, the R, G, and B signals of the corresponding address are read from the image memory 2, converted into L, u, and v signals by the chromaticity matrix circuit 3, and guided to the red-eye correction table 37 (# 202, # 21).
0). Next, for the luminance signal L, # 204 to # 20
6 are executed, and # 212 to # 212 are applied to the color signals U and v.
The process of # 218 is executed.

For the luminance signal, a detection area (L ± d) taking into account an allowable range is designated, and the contents are described in LUT4.
3 (# 204, # 206). On the other hand, the color signal is once converted to polar coordinates (r, φ), and after a detection area (r ± α, φ ± β) is specified in consideration of an allowable range, the inverse conversion (u ₁ ≦ u ≦) is performed. u ₂ , v ₁ ≦ v ≦ v ₂ ), and the contents are written to the LUT 53 (# 212 to # 212).
# 216).

Next, based on the address control signal from the microcomputer 74, the image signals are sequentially read from the image memory 2, and the L, u, v
The signals are converted into signals and guided to the LUTs 43 and 53 (# 2
08, # 218). In the LUTs 43 and 53, L, u, v
The signals are detected in advance in the detection area (L ± d), (u
₁ ≦ u ≦ u ₂ , v ₁ ≦ v ≦ v ₂ ), “1” is output, otherwise “0” is output and output to the AND circuit 121, and The product is taken and sequentially written to the binary data memory 13 (# 22
0).

At the time of reading from the image memory 2 in the next cycle, reading is performed in synchronization with the contents of the binary data memory 121, and when "1" is output from the binary data memory 121, it is regarded as a red-eye position. And switch SW10,
SW11 is switched to a state opposite to the state shown in the figure,
18 to the chromaticity matrix circuit 3 to output "0"
Is output, the switches SW10 and SW11 are switched to the illustrated state, and the output of the image memory 2 is led to the chromaticity matrix circuit 3 as it is.

Next, FIG. 21 is an overall block diagram showing an eighth embodiment of the red-eye correction device, and FIG. 22 is a diagram showing an algorithm of the red-eye position detection processing. Note that, in the drawings, those denoted by the same reference numerals as those in FIGS. 9 and 11 perform the same functions.

In this embodiment, for a series of consecutively shot frames, only the region near the position determined to be the red-eye position in the first frame is treated as the region where the red-eye position is to be detected from the next frame. Thus, the detection of the red-eye position is facilitated and speeded up.

In the figure, the microcomputer 75 takes in the continuous shooting information and the photographing magnification β of a series of frames which are input from the imaging section 20 or manually. The LPF 83 has a filter effect that can be changed according to the photographing magnification β. The circuit block shown in FIG. 21 includes the red-eye position detection circuit section shown in the second embodiment or the third embodiment, and a certain frame is stored in the binary data memory 13 by the red-eye position detection circuit section. It is assumed that the binary data of the detected red-eye position at has been captured.

Next, the red-eye detection processing will be described with reference to FIG. If the microcomputer 75 determines that a series of frames continuously shot (# 230), the binary data stored in the binary data memory 13 remains in the binary data memory 13 even when the playback of the first frame of the continuous shooting is completed. It is kept as it is. Next, by the time the reproduction process for the second frame of continuous shooting is started, it is confirmed whether or not red eyes existed in the first frame of continuous shooting (# 23).
2) If red-eye does not exist in the first frame of continuous shooting, the red-eye position detection processing according to the second and third embodiments is performed for the second and subsequent frames, for example.

On the other hand, if red-eye exists in the first frame of the continuous shooting, only the area near the confirmed red-eye position is extracted and detected for the second and subsequent frames (# 234). That is,
The red-eye position data of the first frame is read from the binary data memory 13, and the area is enlarged via the LPF 83 in accordance with the photographing magnification β. Then, an area signal in which data “1” is generated is guided to one input terminal of the AND circuit 121 also in the vicinity area so as to surround the red-eye position confirmed in the first frame. This is because it is expected that the larger the shooting magnification β is, the larger the movement of the eye position between the first frame and the second and subsequent frames is, and accordingly the eye part is reliably included in the detection area. Is to do so. The data “1” indicating the red part in the frame output from the LUT 52 via the red detection table is input to the AND circuit 12.
1 to the other input terminal.

Then, both data are ANDed by the AND circuit 121 (# 236), output to the microcomputer 75 and stored. Further, the red-eye position detection is performed based on the data of the detected red-eye position in the first frame from the third frame to the last frame in the continuous shooting mode as described above.

In this embodiment, the description has been made based on the binary data in the binary data memory 13. In the fourth to sixth embodiments, the same applies based on the data of the red-eye position taken in the microcomputer. Can also be performed. Also,
The configuration of the first embodiment can be adopted as the configuration of the red-eye correction processing unit.

Next, a ninth embodiment of the red-eye correction device will be described. The red-eye position detection in the ninth embodiment performs red-eye correction by directly specifying the red-eye position manually on the reproduced image on the monitor. That is, the coordinate position (x, y) designated manually is detected, and the detected position data is enlarged via the low-pass filter (x
± Δx, y ± Δy) to ensure detection accuracy, and the LUT 5 (or LUT 5
The red eye position is detected by calculating the logical product of the red part output via the red detection table of (1, 52). At this time, the enlargement range may be made variable according to the imaging magnification β. As shown in FIG. 18, the manual instruction is performed by disposing a transparent touch panel that outputs the pressed position coordinates (x, y) on the monitor screen, or moving the touch panel by operating the operation unit. The position may be designated by operating a display marker on the screen that can be instructed.

Further, a logical product of the ninth embodiment and the seventh embodiment may be calculated (# 222 in FIG. 19) to detect the red-eye position with higher accuracy.

[0074]

As described above, according to the present invention,
A first extracting unit extracts at least one of a low chroma region and a low illuminance region from image data obtained by the imaging unit, a second extracting unit extracts a skin color region, and a third extracting unit extracts a red color region. And the red-eye position in the image is detected using the extracted signals from the first, second and third extracting means, that is, using a plurality of color components specific to the face. , Reliable red-eye position detection is possible.

The red-eye position data obtained in this way can be easily used for color correction for the red-eye position by, for example, reading out the image data synchronously when the image data is read out to a monitor or the like. Become.

[Brief description of the drawings]

FIG. 1 is an overall block diagram showing a first embodiment of a red-eye correction device to which the present invention is applied.

FIG. 2 is a perspective view illustrating an overall structure of an imaging unit.

FIG. 3 is a diagram for explaining a red-eye position.

FIGS. 4A to 4J are diagrams for explaining image processing contents for each detection procedure of a red-eye position.

FIG. 5 is a diagram illustrating an algorithm of red eye position detection and color correction processing.

FIG. 6 is a diagram showing an algorithm of a process A in detail.

FIG. 7 is a diagram showing an algorithm of processing B in detail.

FIG. 8 is a diagram showing an algorithm of processing C in detail.

FIG. 9 is an overall block diagram illustrating a second embodiment of the red-eye correction device.

FIG. 10 is a diagram illustrating an algorithm of a red-eye correction process according to the second embodiment.

FIG. 11 is an overall block diagram showing a third embodiment of the red-eye correction device.

FIG. 12 is a diagram illustrating an algorithm of a red-eye position detection process in a third embodiment.

FIG. 13 is an overall block diagram illustrating a fourth embodiment of the red-eye correction device.

FIG. 14 is a diagram illustrating an algorithm of a red-eye position detection process according to a fourth embodiment.

FIG. 15 is an overall block diagram showing fifth and sixth embodiments of the red-eye correction device.

FIG. 16 is a diagram illustrating an algorithm of a red-eye position detection process according to a fifth embodiment.

FIG. 17 is a diagram illustrating an algorithm of a red-eye position detection process according to a sixth embodiment.

FIG. 18 is an overall block diagram showing a seventh embodiment of the red-eye correction device.

FIG. 19 is a diagram illustrating an algorithm of a red-eye position detection process in a seventh embodiment.

FIG. 20 is a diagram for explaining conversion from a signal (L, u, v) to a luminance system (L) and a polar coordinate system (r, φ). FIG. (r, φ) signal is formed, and FIG. 7B shows a case where the luminance signal L is extracted.

FIG. 21 is an overall block diagram showing an eighth embodiment of the red-eye correction device.

FIG. 22 is a diagram illustrating an algorithm of a red-eye position detection process according to the eighth embodiment.

[Explanation of symbols]

Reference Signs List 1 A / D converter 2 Image memory 3 Chromaticity matrix circuit 4,41,42,5,51,52,53,6,61,6
2 LUT 7, 71, 72, 73, 74, 75 Microcomputer 8, 81, 82, 83, 9, 10 LPF 11 INV 12, 121 AND circuit 13 Binary data memory 14 Matrix circuit 15 Encoder 16 D / A converter 17, Reference Signs List 18 Multiplication unit 19 Monitor 20 Imaging unit 31, 34 Contour extraction unit 32, 35 Shape identification unit 33 Labeling unit 36 Filling unit 37 Red-eye correction table SW1 to SW11 Switch

Continuation of front page (56) References JP-A-5-19382 (JP, A) JP-A-5-224271 (JP, A) JP-A-6-75305 (JP, A) (58) Fields investigated (Int) .Cl. ⁷ , DB name) G03B 27/72-27/80 G06T ⁷ /00-7/60

Claims (1)

(57) [Claims] An image pickup means and an image pickup means obtained by the image pickup means
First extracting means for extracting at least one low saturation region and the low intensity region of the image data, a second extraction means for extracting a skin color area from the image data, red area from the image data <br/> A third extracting means for extracting
Eye position detecting device characterized by comprising a detecting means for detecting a red eye position in the image using signals from the second and third extraction means.