JPH11288459A - Method and device for detecting area such as face and observer tracking display - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an efficient method which can be applied to a wide range of illumination conditions at the time of detecting the candidate object of an area such as a face in a color image. SOLUTION: In a method for detecting an area such as a face in a color image from a video camera, chroma components are obtained from a color image 20 by conversion from an RBG format typical to the output of the video camera (21). The spatial resolution of a chroma component image is deteriorated by spatial averaging (22), and the resolution of the image of a face is deteriorated in each size, and the level is limited so that 5 or less image elements can be obtained. Then, an image whose resolution is deteriorated is tested (23), and an area with substantially uniform chroma, and a prescribed size and shape which is surrounded by an area with substantially non-uniform chroma are obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for detecting a face-like region of a color image. Such a method is used in conjunction with other methods for detecting faces in an image and capturing a target image, for example, at the initialization stage of an image tracking system that may be associated with an observer tracking autostereoscopic display. Can be done. Such methods and apparatus find wide application in, for example, skin color detection, face detection and recognition, security surveillance, video and image compression, video conferencing, multimedia database searching, and computer games.

[0002] The invention also relates to an autostereoscopic observer tracking display.

[0003]

2. Description of the Related Art An autostereoscopic display has a viewer,
Observing such a display with the eyes in two viewing windows makes it possible to see two individual images forming a stereo pair. Examples of such displays are EP 0602934, EP 0656555, EP 0708351,
EP 0726482 and EP 082
No. 9743. FIG. 1 of the accompanying drawings shows an example of a known type of observer tracking autostereoscopic display.

The display is a tracking system 2
And a display system 1 which cooperates with. The tracking system 2 has a tracking sensor 3 that provides a sensor signal to a tracking processor 4. The tracking processor 4 obtains an observer position data signal from the sensor signal, and the observer position data signal is provided to a display control processor 5 of the display system 1.
The processor 5 converts the position data signal into a window control signal, and converts the signal into a window control signal (trac).
ked) is given to the control mechanism 6 of the 3D display 7. In this way, the viewing window for the observer's eyes is steered to follow the movement of the observer's head, keeping the observer's eyes in the proper viewing window within the operating range.

[0005] EP 0 877 274 and GB 2 324 428 disclose low latency, updat for observer tracking autostereoscopic displays.
e) Disclose an observer video tracking system with high frequency and appropriate measurement accuracy. Figure 2 of the accompanying drawings
Shows an example of the system. In this system, a tracking sensor 3 has a Sony XC999 NTSC video camera operating at a field rate of 60 Hz, a tracking processor 4 is provided with a mouse 8,
Indy series silicon graphics entry level where processor 4 is equipped with an R4400 processor operating at 150 Mhz and a video digitizer and frame store with a resolution of 640 × 240 picture elements (pixels) for each field captured by camera 3 It differs from the system shown in FIG. 1 of the accompanying drawings in having a machine. The camera 3 is arranged above the display 7 and faces the observer sitting in front of the display. The normal distance between the observer and the camera 3 is about 0.85 meters, at which distance the observer is about 450 mm in the horizontal or X direction.
Within, it has freedom of movement. The distance between two pixels in the image formed by the camera is approximately 0.67 mm and 1.21 mm in the X and Y directions, respectively.
It is. The Y resolution is halved because each interlaced field is used individually.

FIG. 3 of the accompanying drawings illustrates in general terms the tracking method performed by the processor 4. The method includes a tracking stage 10 following the initialization stage 9. In the initialization stage 9, a target image or “template” is captured by storing a part of the image from the camera 3. The target image generally includes an area of the observer's eye indicated by reference numeral 11 in FIG. 4 of the accompanying drawings. As soon as the target image or template 11 has been successfully captured, observer tracking is performed on the tracking stage 10.

A global target or template search is performed in step 12 to find the position of the target image in all images generated by the camera 3. Once the target image has been found, motion detection is performed in step 13
, Followed by a local target or template search in step 14. The template matching steps 12 and 14 are performed by correlating the target image in the template with each subsection on which the template is overlaid. The best correlation value is compared to a predetermined threshold, and step 15 checks if tracking has been lost. If tracking is lost, control returns to global template matching step 12. If tracking has not been lost, control returns to step 13.

[0008] The motion detection 13 and the local template matching 14 form a tracking loop, and the tracking loop is performed as long as tracking is maintained. The motion detection step includes a difference method (differen
The position data is provided by the TIAL method. The difference method determines the motion of the target image between successive fields and adds this to the location found by the local template matching in the previous step for the previous field.

The initialization stage 9 obtains a target image or template of the observer before tracking starts. European Patent No. 0877274 and British Patent No. 2
The initialization stage disclosed in 324428 uses an interactive method. In the interactive method, the display 7
Displays an input video image, for example, an image generator implemented in processor 4 generates a border image or graphical guide 16 on a display, as shown in FIG. 5 of the accompanying drawings. For example, mouse 8
The user-operable controls that form part of the B. allow for manually driving capture of an image area within the border image.

The observer observes his or her own image on the display 7 together with the border image of the required template size. The observer aligns the midpoint between his eyes with the center line of the graphical guide 16 and activates the system to capture the template, for example, by pressing a mouse button or keyboard key. Alternatively, this alignment may be accomplished by using the mouse 8 to drag the graphical guide 16 to a desired location.

An advantage of such an interactive template capture technique is that the observer can select a template with acceptable registration accuracy.
This includes recognizing a human face and selecting a target image area, such as an eye area. For human vision, this process is straightforward, but such template captures can be used for all possible types with different ages, genders, eye shapes, and skin colors under various lighting conditions. Given the people, computers are difficult.

Suwa et al., "A Video Qual."
ity ImprovementTechnique
for Video Phone and Video
Conference Terminal ", IEEE
Works on Visual Signa
l Processing and Communic
, Sept. 21-22, 1993, Melbourne, Australia, discloses a technique for detecting facial regions based on a statistical model of skin color. This technique assumes that the color and brightness of the face area are in a defined area, and that the face occupies a predetermined amount of space in the video frame. A face region can be found by searching for a color region consisting of pixels whose color is within the region and whose size is within a known size. However, the color space range for skin color changes with changes in light source, direction and brightness. The color space also changes for different skin colors. Therefore, this technique has limited applications because it requires calibrating the skin color space for each particular application and system.

[0013] Swain et al., "Color Index"
ing ", International Journa
l of Computer Vision, 7: 1,
Pages 11-32, 1991, disclose the use of color histograms of multicolored objects to provide color indexing in large model databases. Next, for example, Sako et al., “Real-Time Facial
-Feature Tracking based o
n Matching Techniquesand
its Applications ", 12 IAPR
International Conference
Minutes of on Patent Recognition, Jerusalem, October 6-13, 1994, II
Vol., Pp. 320-324, a technique known as "histogram back projection" is used to find the location of a known object, such as a facial region. However, this technique requires knowledge about the desired target, such as a facial color histogram, and works only if enough pixels in the target image are different from pixels in other parts of the image. Thus, there is a need to provide a controlled background and a need for additional techniques to accommodate changes in lighting.

[0014] Chen et al., "Face Detecti."
on by Fuzzy Pattern Match
ing ", IEEE (0-8186-7042-8),
591-596, 1995, use a fuzzy pattern matching method based primarily on the extraction of skin color using a model known as the "skin color distribution function" (SKDF), to obtain a face-like image in the input image. It discloses a technique for detecting a unique area. This technology first uses Wyszec
hi et al., "Color Science", John.
Wiley & Sons Inc. Converts RGB to Farnsworth color space as disclosed in 1982. An SCDF is constructed by collecting a large set of sample images containing a human face and selecting a skin area in the image by a human viewer. Next, the frequency of each color in the color space appearing in the skin region (frequency)
To find out, a learning program is applied. next,
The SCDF is unified and used to estimate how similar the color is to the skin color. Once the region is extracted as likely to be a skin region, the region is further analyzed based on a pre-established facial shape model, each containing 10 × 12 square cells. However, a problem with this technique is that the SCDF can change with changes in lighting conditions.

[0015]

In the prior art as described above, under various lighting conditions, different ages, genders, eye shapes,
For various types of people with different skin colors, it has been difficult to find facial candidates in color images.

Accordingly, it is an object of the present invention to provide a simple method and apparatus that can be applied over a wide range of lighting conditions without the need for color calibration, is more reliable than known techniques, and has significantly reduced computational requirements. To provide. Also,
It is an object of the present invention to provide a method and an apparatus capable of recognizing face candidates in images of people of different ages, genders, and skin colors, for example, even when wearing brightly colored glasses. It is a further object of the present invention to provide a method and apparatus which is very efficient, realized in real time and can be used for low cost commercial applications. It is another object of the present invention to provide an observer tracking display including a device for detecting a face as described above.

[0017]

According to a first aspect of the present invention, there is provided a method for detecting a face-like region of a color image, wherein a resolution of the color image is reduced by averaging saturation. Forming a reduced-resolution image; and coloring a portion of the reduced-resolution image that is a region of the reduced-resolution image having a predetermined shape and that surrounds the predetermined shape. Retrieving regions of the reduced resolution image having substantially uniform saturation that is substantially different from the degree.

[0018] The color image may include a plurality of picture elements, and the resolution may be reduced so that the predetermined shape extends over two or three resolution-reduced picture elements.

The color image includes a rectangular array of M × N picture elements, and the reduced-resolution image is (M / m)
× (N / n) picture elements, each corresponding to the m × n picture element of the color image, and the saturation of each picture element of the image with reduced resolution is represented by the following formula:

[0020]

(Equation 2)

Where f (i, j)
Is the saturation of the picture element in the i-th column and j-th row of the m × n picture element (32). The method may include storing the saturation in a store.

By comparing the saturation of each of the reduced resolution picture elements with the saturation of at least one adjacent reduced resolution picture element, a uniform value is assigned to each of the reduced resolution picture elements. May be assigned.

If the following equation is satisfied, a first value is assigned to each uniform value: (max (P) -min (P)) / max (P) ≤T where max (P) and max (P) (P) is the maximum and minimum of the saturation of the reduced-resolution picture element and the or each of the adjacent picture elements, respectively, and if T is a threshold and the equation is not satisfied, the first value Is assigned to each uniform value. T may be substantially equal to 0.15.

A uniform value may not be assigned to the or each adjacent reduced-resolution picture element, and each uniform value may be stored in the store.

The resolution is reduced so that the predetermined shape spans two or three reduced resolution picture elements;
The method further comprises the step of determining that the uniform value of the first value is one reduced resolution picture element, two vertically or horizontally adjacent reduced resolution picture elements, and a 2 × 2 pixel picture element rectangle.
Assigning to any one of the arrays, wherein the uniform value of the second value is indicative of the detection of a face-like region when assigned to each of the surrounding reduced resolution picture elements. Good.

The or each adjacent reduced-resolution picture element is not assigned a uniform value, each uniform value is stored in the store, and the detection is a third value different from the first and second values. The value may be indicated by storing it in the store.

[0027] The search step may include performing at least one search in a state where the resolution is reduced and the picture element whose resolution has been reduced is shifted with respect to the color image picture element.

Saturation is obtained from the red, green and blue components by the following equation (21): (max (R, G, B) -min (R, G, B)) / m
ax (R, G, B) where max (R, G, B) and min (R, G,
B) are the maximum and minimum values of the red, green and blue components, respectively.

[0029] The method may include the step of capturing the color image.

[0030] The capturing step may include that the color image is captured by a video camera, and the resolution reducing and searching steps may be repeated for different video fields or frames from the video camera.

In the capturing step, a first color image is captured while illuminating the expected area of the face position, a second color image is captured using ambient light, and the second color image is captured using ambient light. A color image may be subtracted from the first color image to form a color image.

According to a second aspect of the present invention, there is provided an apparatus for detecting an area such as a face of a color image, wherein the resolution of the color image is reduced by averaging the saturation. Forming a reduced image, wherein the region of the reduced resolution image having a predetermined shape is substantially different from the saturation of a portion of the reduced resolution image surrounding the predetermined shape. An apparatus is provided that includes a data processor arranged to search for regions of the reduced resolution image having uniform saturation.

According to a third aspect of the present invention there is provided an observer tracking display including an apparatus according to the second aspect of the present invention.

The operation will be described below.

It is known that human skin tends to have a uniform saturation. The method and apparatus of the present invention take advantage of this property to provide an effective way to find facial candidates in color images. This technique is more reliable and convenient than known techniques, since a wide range of lighting conditions are accommodated without the need for color calibration. By reducing the saturation resolution of the image, the computational requirements are greatly reduced and a relatively simple method can be used. Because averaging improves the uniformity of saturation in the face area, this technique can recognize facial candidates in images of people of different ages, genders, and skin colors, and use bright colored glasses. We can cope when we are running. This technique is so efficient that it can be implemented in real time and used for low cost commercial applications.

This technique can be used in the early stage 9 shown in FIG. 3 of the accompanying drawings for the image tracking system disclosed in EP 0 877 274 and GB 2 324 428. Further, this technique is described, for example, in U.S. Pat. No. 5,169,492, U.S. Pat. No. 5,012,522, Turk et al., "Eigenfa.
ces for Recognition ", Jour
nal 1 of Cognitive Neurosc
ence, Vol. 1, No. 1, pp. 70-86, 1991,
Yuille et al., "Feature Extracti
on from Faces using Defor
mable Templates ", Internet
ionical Journal of Computer
Vision, 8 (2), 99-111, 199
2 years, and Yang et al., Human Face De
protection in Complex Backgr
sound ", Pattern Recognition
n, Vol. 27, No. 1, pp. 53-63, 1994, can be used as the first part of a two-stage face detection and recognition technique. In such a two-stage technique, the first stage finds the approximate location of the face, and the second stage further analyzes each candidate's face region to confirm the presence of the face, eyes, nose and lips. Extract accurate faces such as. The first stage does not require high precision and can be implemented with a fast algorithm. The number of image areas that must be analyzed in the second stage is limited in the first stage. This is advantageous. This is because the second stage generally requires more complex algorithms and is therefore more computationally intensive.

[0037]

BRIEF DESCRIPTION OF THE DRAWINGS The invention will be further explained by means of embodiments with reference to the accompanying drawings. It should be noted that like reference numerals refer to like parts throughout the drawings.

FIG. 6 is a flowchart illustrating a method for automatically detecting and finding a face-like region of a pixelated color image from a video image sequence. The video image sequence may be provided in real time, for example, by a video camera of the type described above with reference to FIG. The method may operate in real time as part of the initialization stage 9 shown in FIG.

In step 20, red, green and blue (RG
B) The latest digital image in the format is obtained.
For example, this step may include storing the latest field of video data from the video camera in a field store. In step 21, the video image is
The RGB format is converted to the HSV format, and the saturation of each pixel is obtained. In practice, it is sufficient to obtain only the S component in step 21, which can be used to overwrite the RGB pixel data or one of its components in the field store and minimize memory requirements. .

The RGB format is a hardware-oriented color scheme that results from the way camera sensors and display phosphors work. The HSV format is one of several formats, including Hue Saturation Luminance (HSI) and Hue Lightness Saturation (HLS).
It is more closely related to the concept of shade and tone. HSV
In the format, hue indicates the color described by the wavelength of light (eg, the distinction between red and yellow),
Saturation is the amount of color present (e.g., the distinction between red and pink), and lightness, luminance or value is the amount of light (e.g., between dark red and light red, or dark gray). Distinction between light gray). The "space" in which these values may be plotted may be shown, for example, as a cone or hexagon or a double cone, as shown in FIG. Here, the conical axis is the progression of the intermediate color from black to white, the distance from the axis indicates saturation, and the direction or angle around the axis indicates hue.

The color of human skin is formed by a combination of blood (red) and melanin (yellow, brown). Skin color exists between these two extreme hues and is somewhat saturated but not very saturated. The chroma component of a human face is relatively uniform.

There are several techniques for converting video image data from RGB format to HSV, HSI or HLS format. Any technique for extracting the saturation component can be used. For example, the conversion may be performed on the saturation component S according to the following equation. When max (R, G, B) = 0, S = 0 When max (R, G, B) is not 0, S = (max (R, G, B) -min (R, G, B))
/ Max (R, G, B)

Following the conversion step 21, the spatial image resolution of the chroma components is reduced by the averaging in step 22. As described above with reference to FIG. 2, the approximate distance of the observer's face from the display is known so that the approximate size of the face in each video image is known. The resolution is reduced so that the adult observer face occupies about 2 to 3 pixels at each dimension shown in FIG.
The technique for achieving this is described in more detail below.

Step 23 is a step in the reduced-resolution image from step 22, which is a region having uniform chroma of a predetermined size and shape, surrounded by regions of reduced-resolution pixels having different chromas. Blobs "are detected. Techniques for accomplishing this are also described in further detail below. Step 24
Detects whether a face candidate or a face-like region has been found. If not found, steps 20 to 24 are repeated. If step 24 confirms that at least one candidate has been found, the uniform blob or the position of each uniform blob detected in step 23 is output in step 25.

FIG. 8 shows the image resolution lowering step 22 in more detail. Reference numeral 30 in FIG. 8A indicates the pixel structure of the image given in step 20. Spatial resolution is shown as an ordered rectangular array of M × N square or rectangular pixels. The spatial resolution is reduced by the averaging and is indicated by reference numeral 31 (M / m) in FIG.
An array of × (N / n) pixels is obtained. The array of pixels 30 comprises pixels 32 each containing m × n pixels of structure 30.
Are effectively divided into "windows" or rectangular blocks. The S value of the pixel is shown in FIG. 8 as f (i, j) (when 0 ≦ i <m and 0 ≦ j <n). The average saturation value P of the window is calculated by the following equation.

[0046]

(Equation 3)

In the embodiment shown in the drawings, the reduced spatial resolution causes the adult observer's face to occupy about two to three of the reduced resolution pixels in each dimension.

Step 23 involves assigning a uniform state or value U to each pixel of reduced resolution and detecting a pattern of uniform values indicative of a face-like region. The uniform value is 1 or 0 depending on the saturation of a pixel and its vicinity.
It is. FIG. 9A shows a pixel having an averaged saturation value P ₀ (reference numeral 35). The uniform value U of the saturation value P ₀ indicated by reference numeral 36 in FIG. 9B is calculated from P ₀ and the averaged saturation values P ₁ , P ₂ and P _{3 of three} neighboring pixels. The uniform value assignment starts at the upper left pixel 37 and proceeds from left to right until the uniform value is assigned to the second pixel 38 from the end of the top row. This process is repeated for each row from top to bottom to the penultimate row. By "scanning" the pixel in this way and using the neighboring pixels to the right and below the pixel for which the uniform value was calculated, the average saturation is overwritten by overwriting the store so that memory capacity can be used efficiently. It is possible to replace the value P with the uniform value U, without having to provide any further memory capacity for the uniform value.

The uniform state U is calculated as follows. U = 1 when (fmax-fmin) / fmax≤T, U = 0 when (fmax-fmin) / fmax> T where T is a predetermined threshold value having a typical value of 0.15, for example. , fmax is the maximum value of _{P 0, P 1, P 2} , and P _3, fmin is the minimum value of _{P 0, P 1, P 2} , and P _3.

Upon completion of the uniform value assignment, array 36
Contains 0 and 1 patterns indicating uniformity of chroma for pixels with reduced resolution. Step 23 looks for a specific pattern of 0s and 1s to detect areas such as faces.
FIG. 10 shows an example of four patterns of uniform values and corresponding pixel saturation patterns, such as face candidates in a video image, corresponding to those patterns. FIG. 10 shows the uniform blob at reference numeral 40. Here, the dark area indicates an averaged saturation value having sufficient uniformity to indicate a face-like area. The surrounding bright areas or squares surround areas of uniform saturation pixels and indicate areas having substantially different saturation. The corresponding pattern of uniform values is indicated by reference numeral 41 and includes pixel locations having a uniform value of 1, completely surrounded by pixel locations having a uniform value of zero.

Similarly, FIG. 10 shows another area such as a face by reference numeral 42 and a corresponding uniform value pattern by reference numeral 43. In this case, two horizontally adjacent pixel locations have a uniform value of 1 and are completely surrounded by pixel locations having a uniform value of 0.
FIG. 10 refers to a third pattern in which the uniform value is indicated by reference numeral 45 and two vertically adjacent pixel locations have a uniform value of 1 and are surrounded by pixel locations having a uniform value of 0. Indicated by reference numeral 44.

The fourth pattern, designated 46 in FIG. 10, represents a square block of 4 (2 × 2) pixel locations having a uniform value of 1 completely surrounded by pixel locations having a uniform value of 0. Have. Therefore,
Whenever any of the patterns of uniformity indicated by reference numerals 41, 43, 45 and 47 in FIG. 10 occur, step 23 indicates that a face-like region or candidate has been found. Searching for these patterns can be performed efficiently. For example, the uniformity of pixel locations is checked sequentially by scanning left to right in each row and top to bottom in the field. When a uniform value of 1 is detected, neighboring pixel locations to the right and below the current pixel location are examined. If at least one of these uniform values is also 1 and the area is surrounded by a uniform value of 0, a pattern corresponding to a possible face candidate is found. Next, the corresponding pixel location may be a uniform value, eg, a value other than 1 or 0 (eg,
2 value). Unless a potential face candidate is found,
The position of the candidate is output.

The appearance of the patterns 40, 42, 44 and 46 can be affected by the actual location of the face-like region relative to the reduced resolution pixel 36 structure. FIG.
Shows an example of a face-like region having a 2 × 2 pixel size with reduced resolution indicated by reference numeral 49. When the area such as the face indicated by the circle 50 is substantially centered on a 2 × 2 block, a pattern 47 having a uniform value is obtained, and the detection is correct. However, if the face is shifted by half a pixel in the horizontal and vertical directions, as indicated by reference numeral 51, the central part of the face-like region is, as indicated by reference numeral 51, the surrounding region. May have different uniform values. This can result in a failure to detect pure candidates.

To avoid such possible problems, steps 21 to 24 can be repeated for the same video field of image data or for one or more consecutive video fields. However, each time steps 21 to 24 are repeated, the position of the reduced resolution pixel array 31 is replaced by the color image pixel array 30.
Change with respect to This is shown in FIG. In FIG. 12, the entire image is denoted by reference numeral 52, and an area used for lowering the spatial resolution by image averaging is denoted by reference numeral 53. Averaging is performed in the same way as shown in FIG. 8, but the starting position changes. In particular, the starting position for the first pixel in FIG. 8 is the upper left corner 54 of the entire image 52, while FIG. 12 shows the subsequent averaging. Here, the starting position is shifted horizontally from the upper left corner by an amount Sx to the right,
It is shifted down by the amount Sy in the vertical direction. Where 0
<Sx <m and 0 <Sy <n.

Each image can be iteratively processed such that all combinations of Sx and Sy are used and an m × n process is performed. However, in practice, not all starting positions need to be used, especially in applications where the detection of regions such as faces does not have to be very accurate. For example,
If the detection of a face-like region forms the first step of a two-step process, as described above, the values of Sx and Sy may be selected from a more sparse set combination as follows. Sx = ix (m / k) and Sy = jx (n / l) where i, j, k and l are integers satisfying the following relationship. 0 ≦ i <k 0 ≦ j <l 1 ≦ k <m 1 ≦ l <n This is a k × l combination in total.

As described above, steps 21 to 24
It can be repeated at different starting positions on the same image or successive images. For real-time image processing, it may be necessary or desirable to repeat the steps for successive images. The method can be performed very quickly, depending on the number of face candidates present in the image, 10 Hz and 60 Hz
Can be done in real time at field rates between Thus, within a short period of time, on the order of only a few seconds or less, all possible locations can be tested.

The method shown in FIG. 6 can be implemented in any suitable hardware, such as the method shown in FIG. As described above, the tracking processor 4 can be programmed to execute the method of FIG. 6 as part of the initialization stage 9 shown in FIG. The data processing is performed by an R4400 processor and associated memory, and the processor 4 generates a saturation value, as shown in FIG.
Includes a video digitizer and frame store to store the averaged chroma values and uniform values of the reduced resolution pixels.

The method shown in FIG. 6 works well with uniform illumination, including ambient light, and can be applied to applications under poor lighting conditions by using an active light source.
Although this method does not require any special illumination and is very quick to recover from changes in the observer's illumination, the initialization stage 9 in FIG. 2 uses an active light source to perform the next observer tracking. It may be switched off during. Tracking is very powerful and does not require special lighting.

FIG. 13 shows a display of the type shown in FIG. 2 modified to provide active illumination. Active light sources include a flashlight 55 having a synchronizer controlled by the processor 4. The flashlight 55 is arranged at an appropriate position, such as on the display 7 and next to the sensor 3, and irradiates the observer's face.

FIG. 14 shows the video tracking system 2, in particular the data processor 4, in more detail. The data processor has a central processing unit (CPU) 56 connected to a CPU bus 57. The system memory 58
It is connected to the bus 57 and contains all system software for operating the data processor.

The video camera 3 has a video digitizer 5
9, a video digitizer 59 includes a data bus 60, a flashlight 55 having a synchronizer,
When the CPU 56 and the optional video display 61 are provided, the video display 61
It is connected to the. The frame store 62 is connected to the data bus 60 and the CPU bus 57.

In embodiments that do not use active lighting, the frame store need only have one field capacity. In the case of the video camera 3 having a field resolution of 640 × 240 pixels, a 24-bit RG
640 × 240 × 3 = 460 for the B color signal
800 bytes capacity is required. For embodiments using active lighting, the frame store 62 is
One field of video data capacity (ie, 9216)
00 bytes).

In use, the flashlight 55 is synchronized with the video camera 3 and the video digitizer 59, and the flashlight is switched on or off at the appropriate times when an image is being captured.

The flash light 55 is used to illuminate the observer's face to increase the uniformity of the distribution. If the flashlight 55 is much stronger than the ambient light,
The luminance of the face is mainly determined by the flash light 55. However, using a strong light source tends to produce supersaturated images, where many objects can be erroneously detected as face-like regions. In addition, the use of powerful flashlights is unpleasant for the observer and can damage the eyes.

Therefore, the flashlight 55 preferably has a medium intensity. In this case, the effects of ambient light may need to be reduced to improve the reliability of detecting regions such as pure faces.

The method shown in FIG. 6 can be modified to compare two consecutive frames of video image data, one obtained with the flashlight 55 illuminated and the other obtained with ambient light only. Thus, the first of these frames contains the effects of both ambient light and flashlight 55. Therefore, this first image I (a + f) can be considered to include two components. I (a + f) = I (a) + I (f) where I (a) is an image obtained only with ambient light,
(F) is an image that would be generated if the only light source was flashlight 55. This can be rewritten as follows. I (f) = I (a + f) -I (a) Therefore, in step 21 or step 22, the effect of the background oversaturation due to the flashlight 55 is reduced by subtracting the image pixel data or the data with reduced resolution. obtain. Further reductions may be obtained by ensuring that the flashlight 55 directs light primarily to areas likely to be occupied by the observer's face.

[0067]

As described above, according to the method and apparatus of the present invention, there is provided a method of finding a face candidate in a color image by utilizing the characteristic that human skin has uniform saturation. This technique is more reliable and convenient than known techniques, since a wide range of lighting conditions are accommodated without the need for color calibration. By reducing the saturation resolution of the image, the computational requirements are greatly reduced and a relatively simple method can be used. This technique works at different ages,
Face candidates in images of people of gender and skin color can be recognized, and it is possible to cope with wearing bright-colored glasses. Because this technology is so efficient,
It can be realized in real time and used for low cost commercial applications.

Such a method according to the invention is, for example,
The initialization stage of an image tracking system, which may be associated with an observer tracking autostereoscopic display, can be used in conjunction with other methods for detecting faces in an image and capturing a target image. Such methods and devices are widely applicable, for example, in skin color detection, face detection and recognition, security surveillance, video and image compression, video conferencing, multimedia database searching, and computer games.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of a known type of observer tracking autostereoscopic display.

FIG. 2 is a schematic block diagram of an observer tracking display to which the present invention can be applied.

FIG. 3 is a flowchart illustrating observer tracking in the display of FIG. 2;

FIG. 4 shows a typical target image ((a)) or template ((b)) captured by the method shown in FIG.

5 shows the appearance of the display during template capture by the display of FIG. 2;

FIG. 6 is a flowchart showing a method for detecting a face-like region constituting an embodiment of the present invention.

FIG. 7 shows a hue saturation value (HSV, hue saturat)
FIG. 4 is a diagram showing an (ion value) color system.

8A and 8B are diagrams showing a reduction in image resolution due to averaging in the method shown in FIG. 6, wherein FIG. 8A shows a given image structure, and FIG. Show.

FIGS. 9A and 9B are diagrams for explaining calculation of a uniform value in the method shown in FIG. 6;

FIGS. 10A to 10D are diagrams showing patterns used for selecting a face candidate in the method shown in FIG. 6; FIG. 5 shows an example of a pixel saturation pattern such as a face candidate.

11A and 11B are diagrams showing the influence of different positions of the face on the method shown in FIG. 6, wherein FIG.
(B) shows a case where a face-like region is shifted by half a pixel in the horizontal and vertical directions.

FIG. 12 illustrates a modification to the method shown in FIG. 6, accommodating different face positions.

FIG. 13 is a schematic block diagram of an observer tracking display to which the present invention is applied.

FIG. 14 is a system block diagram of the video tracking system of the display of FIG. 13 for implementing the method of the present invention.

[Explanation of symbols]

 Reference Signs List 1 display system 2 tracking system 3 tracking sensor 4 tracking processor 5 display control processor 6 control mechanism 7 3D display with tracking function 8 mouse 9 initialization stage 10 tracking stage 11 template

 ──────────────────────────────────────────────────の Continuing on the front page (72) Inventor David Ezra UK 100 ARL Oxfordshire, Wallingford, Brightwell-Kam-Sotwell, Monksmead 19

Claims (17)

[Claims] 1. A method for detecting a face-like region of a color image, the method comprising the step of reducing the resolution of the color image by averaging the saturation, forming an image having a reduced resolution. A step of substantially uniform saturation in a region of the reduced resolution image having a predetermined shape, the saturation being substantially different from a saturation of a portion of the reduced resolution image surrounding the predetermined shape. Retrieving regions of the reduced resolution image having: 2. The method of claim 1, wherein the color image includes a plurality of picture elements, and the resolution is reduced such that the predetermined shape spans two to three reduced resolution picture elements. . 3. The color image includes a rectangular array of M × N picture elements, and the reduced-resolution image is (M / N
m) × (N / n) picture elements, each corresponding to the m × n picture element of the color image, and the saturation P of each picture element of the image with reduced resolution is represented by the following equation: 1) Where f (i, j) is the i of the m × n picture element
3. The method according to claim 2, wherein the saturation of the pixel in the jth column and the jth row is the saturation. 4. The method according to claim 3, comprising storing the saturation in a store. 5. Comparing the saturation of each of the reduced-resolution picture elements with the saturation of at least one adjacent reduced-resolution picture element, 4. A uniform value is assigned.
Or the method of 4. 6. A first value is assigned to each uniform value if the following equation is satisfied: (max (P) −min (P)) / max (P) ≦ T, where max (P) And max (P) are the maximum and minimum of the saturation of the reduced-resolution picture element and the or each adjacent picture element, respectively, if T is a threshold and the equation is not satisfied, The method of claim 5, wherein the value is assigned a second value different from the first value. 7. The method of claim 6, wherein T is substantially equal to 0.15. 8. The method of claim 1, further comprising the step of storing the saturation in a store, wherein the or each adjacent reduced-resolution picture element is not assigned a uniform value, and in the storing step, each uniform value is assigned to the corresponding color saturation. The method according to any one of claims 5 to 7, wherein the method is stored in the store instead of the degree. 9. The method of claim 1, wherein the resolution is reduced such that the predetermined shape spans two or three reduced resolution picture elements, and the method further comprises:
Two reduced resolution picture elements, two vertically or horizontally adjacent reduced resolution picture elements, and a picture element rectangle 2
Assigning to any one of the x2 arrays, wherein the uniform value of the second value is assigned to each of the surrounding reduced resolution picture elements, including indicating detection of a face-like region. A method as claimed in claim 6 or claim 7. 10. The method according to claim 10, further comprising the step of storing the saturation or the uniform value in a store, wherein in the storing step, a value different from the first and second values is substituted for the corresponding saturation or uniform value. 10. The method of claim 9, wherein a value of 3 is stored in the store, wherein the storage indicates a detection. 11. The search step includes repeating the resolution reduction, and performing at least one search in a state where the reduced-resolution picture element is shifted with respect to the color image picture element. 11. The method according to any one of 2 to 10. 12. The saturation is obtained from the red, green and blue components by the following equation: (max (R, G, B) −min (R, G, B)) / m
ax (R, G, B) where max (R, G, B) and min (R, G,
B) is the maximum and minimum of the red, green and blue components, respectively.
The method described in one. 13. The method according to claim 1, comprising capturing the color image. 14. The capturing step includes that the color image is captured by a video camera, wherein the resolution reducing step and the searching step include:
14. The method of claim 13, wherein the method is repeated for different video fields or frames from the video camera. 15. In the capturing step, a first color image is captured while illuminating an expected area of the face position, and a second color image is captured using ambient light. 15. The method of claim 14, wherein two color images are subtracted from the first color image to form a color image. 16. An apparatus for detecting a face-like region of a color image, wherein the resolution of the color image is reduced by averaging the saturation, and an image having a reduced resolution is formed. A region of the reduced-resolution image having a shape having a substantially uniform saturation that is substantially different from the saturation of a portion of the reduced-resolution image surrounding the predetermined shape. An apparatus having a data processor arranged to search for regions of an image with reduced image quality. 17. An observer tracking display comprising the device according to claim 16.