KR20140084675A - Multiple Human Face Detection Method And Electronic Device supporting the same - Google Patents

Abstract

The present invention relates to multiple human face detection. The present invention provides a multiple human face detection method, which comprises the steps of collecting a thermal image and a charge-coupled device (CCD) image of a predetermined region; generating a matched image by matching the thermal image based on the CCD image; calculating candidate regions higher than a predefined reference temperature from the generated matched image; and performing detection of human faces from the candidate regions, and an electronic device supporting the same. According to the method of the present invention, multiple human faces can be accurately detected by fusing the thermal image and the CCD image, and the multiple human faces can be detected even in an image with a complex background. Also, the detection speed for face detection is high, and system reliability can be improved due to a low incorrect detection rate.

Description

TECHNICAL FIELD [0001] The present invention relates to a multi-user face detection method and a terminal device supporting the same,

The present invention relates to multi-user face detection, and more particularly, to a multi-user face detection method for supporting rapid detection of a face region of a user by fusing image information of a thermal camera and a CCD camera and a terminal device supporting the same .

Recently, as the interest in the system for monitoring the user has increased, research and development of a device for detecting a user 's face in a dynamic image in real time has been actively carried out. Especially, in the intelligent system that monitors the user, the face area is accurately and quickly detected, so that speed and accuracy are emphasized because the performance and stability can be ensured in the subsequent application range.

In order to detect the conventional face region, all regions in the image are detected. In such a case, if the texture and the background element of the image are complicated, the area other than the face is also erroneously detected as a face, and the detection rate is deteriorated. This problem has resulted in a problem of lowering the reliability of the system. In addition, in a real-time surveillance system that detects a face in real time to determine a face area, when a face area is determined for all areas in an image, there is a problem that the face area detection and determination are not normally performed for a fast moving user. Particularly, face and shape of user have problems in proper face detection because color, shape and background may be distorted depending on environment change, movement type, and lighting environment.

Accordingly, it is an object of the present invention to provide a multi-user face detection method and a terminal device supporting the multi-user face detection method that can accurately detect a face of a user and provide highly reliable information.

It is another object of the present invention to reduce the system load by detecting a face of a user by selecting a candidate region by merging a thermal image and a CCD image without judging all areas of the image when detecting a user's face, And a terminal device supporting the multi-user face detection method.

According to an aspect of the present invention, there is provided an image processing apparatus including an infrared camera for collecting a thermal image, a CCD camera for collecting CCD images in the same area as the area captured by the thermal camera, And a control unit for controlling the candidate regions to detect a face of a person in the candidate regions by calculating candidate regions that are higher than a predefined reference temperature in the generated matching image, A configuration of a terminal device supporting user face detection is disclosed.

Wherein the control unit includes a thermal image candidate region generation unit for determining an average temperature of temperature values of all the pixels collected by the thermal image in the matched image based on the CCD image as the reference temperature, An upper half body detecting unit for performing upper half body detection by applying Haar like feature detection having a defined pattern, and a face detecting unit for applying Haar like feature detection to apply a predefined pattern for face detection.

The present invention also relates to a method and system for capturing an image, comprising the steps of: collecting an infrared image and a CCD image for a region, generating a matching image by matching the thermal image with the CCD image, Calculating candidate regions above the temperature, and performing face detection of the person in the candidate regions.

The step of generating the matched image includes determining an average temperature of all the temperature values of all the pixels collected by the thermal image in the matched matched image with reference to the CCD image as the reference temperature.

The method may further include performing upper half detection by applying Haar like feature detection having a predefined pattern to the candidate regions after calculating the candidate regions, And applying Haar like feature detection applying a predefined pattern for face detection.

According to the multi-user face detection method and the terminal device supporting the multi-user face detection method according to the embodiment of the present invention, it is possible to accurately detect faces of multiple users by merging a thermal image and a CCD image, The user's face can be detected.

Further, in the present invention, the detection speed is high in the face detection, and the false detection rate is reduced and the system reliability can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of a terminal device supporting a multi-user face detection function according to an embodiment of the present invention; FIG.
2 is a detailed block diagram of a control unit in a terminal device according to an embodiment of the present invention;
3 is a diagram for explaining a multi-user face detection method of the present invention.
4 is a diagram for explaining a candidate region setting during multi-user face detection according to the present invention.
5 is a diagram for explaining a result of applying multi-user face detection of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, only parts necessary for understanding the operation according to the embodiment of the present invention will be described, and the description of other parts will be omitted so as not to disturb the gist of the present invention.

The terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary meanings and the inventor is not limited to the meaning of the terms in order to describe his invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention, so that various equivalents It should be understood that water and variations may be present.

1 is a diagram schematically illustrating a configuration of a terminal device 100 supporting multi-user face detection according to an embodiment of the present invention.

Referring to FIG. 1, a terminal device 100 supporting multi-user face detection according to the present invention includes a thermal imaging camera 110, a CCD camera 120, a display unit 140, a storage unit 150, As shown in FIG. In the terminal device 100 according to the embodiment of the present invention, the configuration of the thermal imaging camera 110 and the CCD camera 120 are described for collecting a thermal image and a CCD image, respectively, but the present invention is not limited thereto . That is, a camera applicable to the terminal device 100 of the present invention may be implemented by employing a fusion sensor structure capable of collecting the thermal image and the CCD image described above.

The terminal device 100 of the present invention fuses the thermal image collected by the thermal imaging camera 110 and the CCD camera 120 with the CCD image and calculates an average temperature value for all image pixels in the fused image , Only the region above the average temperature value is set as the candidate region. Then, the terminal device 100 of the present invention can support the detection of the upper half of the set candidate region and the face detection to perform real-time face detection in the photographed region. In particular, since the terminal device 100 performs upper body detection and face detection on candidate regions having an average temperature or higher, the face detection is performed only in a certain region of the entire region. Accordingly, the terminal device 100 of the present invention can support the detection of the user's face in the real-time collected image.

The thermal imaging camera 110 is a device that detects infrared heat energy emitted from an object and displays the thermal energy. This thermal imaging camera 110 has an advantage in that it has less influence on light intensity than other cameras. Various methods can be applied to the thermal imaging camera 110. For example, Thermoelectric Cooling (TE Type) method using Peltier effect in which a temperature difference occurs when current flows between different metals, Joule-Thomson effect generating temperature difference by using pressure difference generated when He gas is concentrated at high pressure and converted to low pressure Argon gas cooling system that emits low-pressure gas to the atmosphere based on the principle similar to Stirling Cooling method (Stirling Cooling) based on Stirling Cooling method; Liquid nitrogen cooling system (LN2) system that uses ambient temperature lowering by liquid nitrogen gasification Etc. may be applied. Also, the thermal imaging camera 110 may employ a non-cooling type sensor. That is, the thermal imaging camera 110 of the present invention may be a camera to which at least one scheme capable of collecting thermal images for a wide area or a predetermined constant area is applied. The thermal image collected by the thermal imaging camera 110 is provided to the control unit 160. In particular, the thermal image collected by the thermal imaging camera 110 can be distinguished from the other regions in which the heat radiating region of the human body is diffused. In addition, the thermal imaging camera 110 may include an area for a heat source that emits a certain infrared ray in addition to the human body.

A charge coupled device camera (CCD camera) 120 is a digital camera that converts an image into an electric signal by using a charge coupled device (CCD), and stores it as digital data in a storage medium such as a flash memory to be. CCD is an optical sensor semiconductor that converts light into electrical signals from a digital camera. The light transmitted through the lens and diaphragm is converted into an electrical signal by the intensity of the light by the CCD. The signal is converted into an image file through an ADC converter which converts the analog signal into a digital signal of 0 and 1 And stored in memory. In particular, the intensity of light incident through the lens and diaphragm is recorded in the CCD, separated into different colors by the RGB color filters disposed adjacent to the CCD, and then converted into electrical signals in a number of photosensitive elements constituting the CCD, . The control unit 160 may control the CCD camera 120 to collect real-time CCD images stored in a memory disposed in the CCD camera 120 for real-time image processing.

The CCD camera 120 of the present invention collects a CCD image in real time in the same area as the area where the thermal imaging camera 110 has photographed the thermal image. At this time, the CCD image may be arranged with at least one human-identifiable object, which may be provided as a temperature image different from the surrounding area in the thermal image.

The display unit 140 is configured to output various screens related to the operation of the terminal device 100 supporting the multi-user face detection function. For example, the display unit 140 may display a thermal image collected by the thermal imaging camera 110 and a CCD image collected by the CCD camera 120, respectively. In addition, the display unit 140 can output an image in which a thermal image and a CCD image are fused. Particularly, the fusion image outputted by the display unit 140 includes a position indication for candidate regions that are higher than an average temperature value in the image, a position indication for regions corresponding to the upper half of the user among the candidate regions, Positional notation for regions may be provided. On the other hand, the display unit 140 may output a menu screen for supporting the multi-user face detection function of the present invention, an item execution screen according to a specific item selection included in each menu, and the like.

The storage unit 150 stores a program, an algorithm, and other image data required to support the multi-user face detection function according to an embodiment of the present invention. For example, the storage unit 150 may temporarily store the thermal image collected and provided by the thermal imaging camera 110, and the CCD image collected and provided by the CCD camera 120. The storage unit 150 may also store a fusion image of the thermal image and the CCD image.

In particular, the storage unit 150 of the present invention may include an average temperature calculation algorithm 151 and a face recognition algorithm 153 for supporting a multi-user face detection function.

The average temperature calculation algorithm 151 includes a routine for matching the coordinates of the thermal image with respect to the CCD image when the thermal image and the CCD image are collected, a routine for collecting temperature values of all the image pixels in the matched image, A routine for calculating an average temperature value with respect to a temperature value of an image pixel; a routine for detecting a candidate region having a temperature value equal to or higher than an average temperature value in all image pixels based on the calculated average temperature value; And may include routines that provide information. The routine for detecting the candidate region may include a routine for detecting, as candidate regions, regions having an area of a predetermined or larger predetermined size among pixels having a temperature equal to or higher than the average temperature value.

The face recognition algorithm 153 is a routine for detecting an area recognized as an upper half of a person based on a predefined upper half Haar like feature among candidate regions calculated according to the average temperature calculation algorithm 151, You can include a face recognition routine using the defined face Haar like feature.

The controller 160 is configured to support processing of various signals required for operation of the terminal device 100 supporting the multi-user face detection function of the present invention, transmission of control signals, supply and distribution of power, and processing and analysis of data . The control unit 160 may include a configuration as shown in FIG. 2 for supporting a multi-user face detection function.

2 is a detailed block diagram of the controller 160 of the terminal device 100 according to the embodiment of the present invention.

2, the terminal device 100 of the present invention includes an image matching processing unit 161, a column image candidate region generating unit 162, an upper body detecting unit 163, And a face detection unit (164).

The image matching processing unit 161 controls to activate the thermal imaging camera 110 and the CCD camera 120 when the function activation for the multi-user face recognition of the present invention is requested. For this purpose, the image matching processing unit 161 may control the power supply and initialization process of the thermal imaging camera 110 and the CCD camera 120. The image matching processing unit 161 collects the thermal image and the CCD image from the thermal imaging camera 110 and the CCD camera 120, respectively, and performs registration of the collected thermal image and CCD images. In particular, the image matching processing unit 161 matches the coordinates of the collected CCD image and the thermal image based on the CCD image. Accordingly, the matching images can be classified or classified according to the pixels based on the CCD image. When the matching image is generated, the image matching processing unit 161 transmits the matching image to the column image candidate region generating unit 162.

The column image candidate region generation unit 162 calculates an average temperature value for all image pixels in the matched image delivered from the image matching processing unit 161. [ To this end, the column image candidate region generation unit 162 calculates the average temperature value by collecting the pixel temperature information of all the images of the matching image and dividing the sum of the pixel temperature information by all the image pixels.

On the other hand, the column image candidate region generation unit 162 supports calculation of candidate regions above the average temperature value based on the calculated average temperature value when the calculation of the average temperature value of the specific frame is completed. For this, the column image candidate region generation unit 162 detects an area above the average temperature value as shown in Equation (1), and designates candidate regions using the detected regions.

[Equation 1]

 The thermal image candidate region generating unit 162 may provide only the regions above the average temperature value as an input image and may process an area below the average temperature value as "0" At this time, regions above the average temperature can be selected as candidate regions for human face recognition. An image in which regions having a temperature equal to or higher than the average temperature are selected as candidate regions is provided to the upper body detecting unit 163.

The upper body detection unit 163 detects an upper body of the multiple users as a region of interest (ROI) on the basis of the image provided with the regions of the average temperature or more provided by the column image candidate region generation unit 162 as candidate regions. At this time, the upper body detecting unit 163 can operate a Haar-like feature detector which is robust against color change due to illumination and can be processed in real time. The upper body detection unit 163 for operating the Haar-like feature detector predefines patterns that can be recognized as an upper body of a person, and detects areas recognized as upper part of a person among the thermal image candidate areas based on the patterns as ROIs. The ROI regions are provided to the face detection unit 164.

The face detecting unit 164 detects the face regions of the end user in the ROI regions provided by the upper body detecting unit 163. The detector used in the face detector 164 is the same Haar-like feature as the detector used in the upper body detector 163. In particular, the Haar-like feature detector used in the face detection unit 164 performs face recognition using a predefined face application pattern. The face detection unit 164 may support the display unit 140 to provide the result of the detection of the face of the multiple users. In addition, the face detecting unit 164 can control the storage of the image being captured when the detection of the face of the multiple users is completed. For example, the face detecting unit 164 can control the storage of the images collected by the CCD camera 120 when the face detection of multiple users is recognized in the images collected by the thermal imaging camera 110 and the CCD camera 120, If there is no multi-user face detection, it is possible to skip the separate image storage. That is, the face detection unit 164 can support application of the detected image results according to various predefined application elements after the face detection.

3 is a diagram for explaining a multi-user face detection method according to an embodiment of the present invention.

Referring to FIG. 3, the multi-user face detection method of the present invention firstly supports the control unit 160 of the terminal device 100 to collect a thermal image and a CCD image in step S101. For this, the controller 160 may control the generation of an input signal for multi-user face detection or activation of the thermal imaging camera 110 and the CCD camera 120 according to predefined schedule information. The control unit 160 may control the thermal imaging camera 110 and the CCD camera 120 to capture an image of an area at a predetermined predetermined angle. To this end, the terminal device 100 may further include a thermal camera 110 and a tilt module for adjusting the angle of the CCD camera 120.

On the other hand, when the thermal image and the CCD image are collected, the control unit 160 performs matching of the images collected in step S103. At this time, the controller 160 may generate a pixel classification matrix for the entire thermal image based on the CCD image. That is, the control unit 160 matches the images so that the entire thermal image can be divided into the pixels of the CCD images.

Next, the controller 160 supports detection of a candidate region of a reference temperature or higher in step S105. In particular, the controller 160 calculates the average temperature value by summing the temperature values of all the image pixels of the matched images matched with the thermal image and the CCD image, dividing the temperature values by the number of all pixels, The above areas can be detected as candidate areas. Here, the control unit 160 may omit the calculation of the average temperature value and may support the application of the specific reference temperature value directly inputted by the manager to the matching image. That is, if there is a predefined reference temperature value, the controller 160 can select a region where pixels over the reference temperature value are distributed as thermal image candidate regions.

In step S107, the controller 160 detects the upper half of the column image candidate regions. For this, the controller 160 applies a Haar-like feature having a predefined pattern that can be recognized as an upper body of a person to the thermal image candidate regions, and only the regions recognized as the upper half of the corresponding thermal image candidate regions are subjected to ROI Area. ≪ / RTI >

If at least one ROI corresponding to a person's upper half of the candidate regions is detected, the controller 160 can perform face recognition using the Haar-like feature detector for the corresponding ROI region in step S109. At this time, the Haar-like feature detector can perform face detection of multiple users by applying a face pattern recognizable to a human face to ROI regions.

In step S111, the control unit 160 may control the multi-user face detection of the present invention to be terminated when an input event or schedule for ending frame collection arrives. If there is no input event for ending frame collection in step S111 and the frame collection is continuously generated in real time, the control unit 160 may branch back to step S101 and perform the following process again.

As described above, the multi-user face detecting method and the terminal supporting the multi-user face detecting method according to the embodiment of the present invention calculate at least one candidate region capable of detecting a user face using a thermal image for a specific image, By applying the Haar-like feature detection to only the candidate regions calculated, it is possible to dramatically reduce the area applied to the comparison detection of the pixel values. Accordingly, the present invention can operate only a very small amount of calculation in performing multi-user face detection, and thus supports real-time multi-user face detection. In addition, since the operation of the relatively small amount of computation enables the hardware of the terminal device 100 to be simplified, it is possible to facilitate the production and production of the terminal device supporting the multi-user face detection function more easily and economically.

4 is a diagram showing the result of the candidate region setting step through the thermal image analysis.

As shown in Fig. 4, coordinate matching is performed on the thermal image collected by the thermal imaging camera 110 on the basis of the CCD image, and only the image above the reference temperature value in the thermal image is calculated based on the matching coordinate can do. As a result, the terminal device 100 of the present invention can detect only whole body image regions corresponding to multiple users as shown in FIG.

5 is a diagram illustrating a result of applying the multi-user face detection function of the present invention.

Referring to FIG. 5, when thermal image and CCD image for the same area are collected, the terminal device 100 matches the thermal image based on the CCD image, and then applies a predetermined reference temperature to each pixel Can be performed. As a result, the terminal device 100 can obtain an image in which only pixels having a temperature equal to or higher than a predetermined temperature, for example, a temperature equal to or higher than the average temperature value in the matching image, are detected as candidate regions.

The terminal device 100 can detect the upper-body regions 141 including the upper half of the human body by applying Haar-like feature detection to the image including the candidate regions. That is, the terminal device 100 can detect the two upper-body regions 141 located at the center as the ROI region as shown in the screen 503.

The terminal device 100 in which the ROI region is detected can detect the face regions 143 in the upper body regions 141 as in the 505 screen by applying Haar-like feature detection prepared for face recognition.

As shown in the figure, the terminal device 100 of the present invention selects candidate regions for detecting a face region of a person regardless of background complexity, and performs only face detection on the candidate regions, And can be quickly detected.

The multi-user face detection method and the terminal device supporting the same according to the embodiment of the present invention described above can detect more accurately than the detection system using only the CCD image, The detection speed is faster than that of the other approaches, and the erroneous detection rate is lowered, so that the overall stability of the system can be guaranteed.

While the present invention has been described with reference to several preferred embodiments, these embodiments are illustrative and not restrictive. It will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit of the invention and the scope of the appended claims.

100: terminal device 110: thermal imaging camera
120: CCD camera 140:
150: storage unit 160:
161: image matching processing unit 162: column image candidate area generating unit
163: upper body detection unit 164: face detection unit

Claims (8)

An infrared camera for collecting infrared images;
A CCD camera for collecting CCD images in the same area as the area photographed by the thermal imaging camera;
A control unit for generating a matching image by matching the thermal image based on the CCD image, calculating candidate regions that are higher than a predefined reference temperature in the generated matching image, and performing detection of a face in a candidate region ;
And detecting the face of the user. The method according to claim 1,
The control unit
A thermal image candidate region generation unit for determining an average temperature of temperature values of all the pixels collected by the thermal image in the matched image based on the CCD image as the reference temperature;
And detecting the face of the user. 3. The method of claim 2,
The control unit
An upper half body detecting unit for performing upper half body detection by applying Haar like feature detection having a predefined pattern to the candidate regions;
And detecting the face of the user. The method of claim 3,
The control unit
A face detector for applying Haar like feature detection to apply a predefined pattern for face detection;
And detecting the face of the user. Collecting an infrared image and a CCD image for a predetermined area;
Matching the thermal image based on the CCD image to generate a matching image;
Calculating candidate regions above a predefined reference temperature in the generated matching image;
Performing face detection of a person in the candidate regions;
Wherein the multi-user face detection method comprises: 6. The method of claim 5,
The step of generating the matching image
Determining an average temperature of the temperature values of all pixels collected by the thermal image in the matched matching image based on the CCD image as the reference temperature;
Wherein the multi-user face detection method comprises: The method according to claim 6,
After the step of calculating the candidate regions
Performing upper body detection by applying Haar like feature detection having a predefined pattern to the candidate regions;
Further comprising the steps of: 8. The method of claim 7,
The step of performing the face detection
And applying Haar like feature detection to apply a predefined pattern to the face detection.

Images