KR100901272B1 - System and method for generating an initial template - Google Patents

Abstract

The initial template generating device according to the present invention, the image input unit for acquiring the image using the image capturing means mounted on the robot, and rotates the robot, and receives the image from the image input unit at a predetermined interval during the movement to register the image And a face information register to generate a user's initial template by detecting a feature part after normalizing the face region in the collected registered image.

As described above, the present invention receives an image while rotating the robot, collects a registered image based on the received image, and then configures an initial template based on the received image, thereby including an image including a face shape change in various environments such as lighting and posture. Initial templates can be used to improve the performance of face recognition in robots that require face recognition in harsh environments.

Robot, user recognition, face recognition, face registration, initial template

Description

SYSTEM AND METHOD FOR GENERATING AN INITIAL TEMPLATE}

The present invention relates to a face registration device and a method required for a robot to recognize a user's face. In particular, an initial template of a user is generated by using feature vectors extracted from an image including various face-forming transformations such as lighting and posture. An apparatus and a method for generating an initial template are described.

The present invention is derived from the research conducted as part of the IT new growth engine core technology development project of the Ministry of Information and Communication [Task management number: 2005-S-033-03, Task name: Development and standardization of embedded component technology for URC].

Face recognition technology is not a new technology, numerous studies have been conducted so far, and it is commercialized in some applications. However, as many studies have revealed, the problem of face recognition is that it is vulnerable to changes in lighting and posture. In particular, face recognition in robots requires recognition in very poor environments, unlike conventional face recognition, which has been mainly studied in terms of security.

Since the robot and the user move freely in any place and interact with each other frequently, the lighting and the user's posture vary widely.

In order to guarantee the performance of the face recognition engine in such an environment, a method of constructing and generating an initial registration template using feature vectors extracted from various face images is required. If the diversity of the initial template is not secured, there is a limit to achieving satisfactory performance with existing face recognizers.

Of course, there is a technology for coping with the shape of the face according to the change of lighting and posture using only the user template generated initially. For example, "AS Georghiades, PN Belhumeur, and DJ Kriegman," From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose, "IEEE Trans. On PAMI, vol. 23, no. 6, Jun. 2001."

Another method is to collect various templates through learning, such as "W. Huang, B. Lee, M. Rajapakse, and L. Li," Face Recognition by Incremental Learning for Robotic Interaction, "IEEE workshop on Multimedia Signal Processing, pp. 280-283, Dec. 2002. "

Most conventional face recognition methods use only user templates generated initially to deal with face shape changes due to changes in lighting and posture, and thus have limitations and are difficult to process in real time due to complex processing. have.

In addition, the method of collecting various templates through learning during the conventional recognition may collect various templates through learning, but if the initial template is not diverse, the learning does not occur due to the failure to pass the set verification condition. In addition, there is a high risk that the system collapses due to incorrect learning, and even when the learning is normally performed, there is a problem in that it does not contribute significantly to the improvement of the actual face recognition performance by selecting only the representative pattern through the simple clustering technique.

As described above, although the configuration of the initial template including various face shape changes is a very important factor that determines the performance of the shape-based face recognition system, there is no research on how to construct the initial template.

Accordingly, the present invention proposes a method of constructing an initial template by collecting images including various face shape changes such as lighting and posture.

In accordance with a first aspect of the present invention, an initial template generating apparatus includes an image input unit for acquiring an image by using an image capturing means mounted on a robot, and rotationally moving the robot from the image input unit at a predetermined interval during the movement. A registration image collecting unit receiving an image and collecting a registration image, and a face information registration unit for generating a user's initial template by detecting a feature part after normalizing the face area in the collected registration image.

As a second aspect of the present invention, the initial template generation method comprises: (a) using an image photographing means mounted on the robot at a predetermined interval during the movement of the robot while rotating the robot from an arbitrary position to a preset final position; Acquiring an image, (b) collecting a plurality of registered images by detecting a face region in the acquired image, (c) normalizing the face region of each of the registered images; d) detecting feature portions in the normalized face region and generating an initial template of the user based on each detected feature portion.

In the present invention, an initial template including a face shape change in various environments such as lighting and posture may be configured.

The present invention receives an image while rotating the robot, collects a registered image based on the received image, and then configures an initial template based on the received image, by using images including face shape changes in various environments such as lighting and posture. Initial templates can be configured to improve the performance of face recognition in robots that require face recognition in harsh environments.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

The initial template generating apparatus according to the present invention generates a user's initial template on the basis of detecting a face by acquiring a continuous image including a face while moving the robot to a specified position by measuring the distance between the robots. It is to configure a high reliability face recognition device.

1 is a block diagram illustrating an initial template generating device of a robot according to a preferred embodiment of the present invention.

Referring to FIG. 1, the initial template generating apparatus includes an image input unit 100, a registered image collecting unit 120, a registered image collecting buffer unit 135, a face information registering unit 140, and a user database 160. Here, the registered image collector 120 includes a face detector 122, a distance measurer 124, a position controller 126, and a robot controller 128, and the face information register 140 includes face normalization 142. , A feature extractor 144, and a template generator 146.

The image input unit 100 obtains a continuous image sequence by using image capturing means such as a camera mounted at a predetermined position of the robot, for example, a head, and provides the image sequence to the registered image collecting unit 120.

The registered image collector 120 may include a face detector 122 that detects a face region of a user in a continuous image sequence provided from the image input unit 100, a distance measurer 124 that measures a distance between the robot and the user; A position controller 126 for controlling the position of the camera, which is an image photographing means, and a robot controller 128 for controlling the movement of the robot based on the distance measured by the distance measurer 124 based on the detected position of the face region. It consists of.

The face detector 122 receives each image from the image input unit 100 to determine whether a face is detected, and determines whether a face area is located at the center of the screen in the detected image. The image data where the area is located is stored in the registered image collection buffer unit 135.

On the other hand, when the face area does not exist in the center of the screen, the position control unit 126 controls the portion where the image capturing means is mounted, for example, the head of the robot.

The process of locating the face area in the center of the image is to perform stable distance measurement in a distance measuring step with a user to be performed later.

The distance measuring unit 124 may be, for example, an ultrasonic or laser sensor that measures a distance between the user and the robot by driving at an arbitrary position of the user, for example, a lower position (chest) of the face. 128 may modify the movement trajectory of the robot in real time.

2 to 5 for a method of capturing an image with a camera, which is an image capturing means mounted on a robot, during the process of collecting the image by the registered image collecting unit 120 having the above configuration, ie, a face registration mode. Will be explained.

2A to 2C are diagrams illustrating a face registration mode of a user, FIG. 3 is a diagram illustrating an example of a user face image in the user registration mode of FIG. 2A, and FIG. 4 is a user face in the user registration mode of FIG. 2B. FIG. 5 is a diagram illustrating an example of an image, and FIG. 5 is a diagram illustrating an example of a user face image in the user registration mode of FIG. 2C.

First, among the face registration modes, the first mode is the user's convenience. As shown in FIG. 2A, the user is in a fixed position and the robot moves. In other words, the user's required action is to look at the camera from a fixed position, and the robot equipped with the camera automatically calculates the distance to the user for collecting images in various environments, and changes the lighting while moving to the designated position, Collect images reflecting posture.

In order to collect the registered image in various environments, the robot automatically calculates the distance between the user and the user using the distance measuring unit 124 and controls the robot controller 128 based on the calculated distance to move to a predetermined position. .

While moving to a predetermined position, the face detector 122 extracts the images reflecting the changed lighting, posture, and the like and stores the extracted images in the registered image collection buffer unit 135. On the other hand, since the change in the resolution of the face according to the distance also has a great influence on the face recognition performance, the robot approaches the user from the remote (3m) to the near (1m) from the remote control (128) control the image input unit 100 The image detected by the image capturing means is input, and the face detector 122 extracts an image in which the face region is located in the center of the received images and stores the image in the registered image collection buffer unit 135.

That is, as the robot moves as shown in FIG. 2A, the collected image is as shown in FIG. 3.

As described above, the face registration mode of FIG. 2A requires little cooperation of the user, whereas in the case of FIG. 2B, the robot controller 128 is a mode requiring cooperation of the user, that is, a robot located at a certain distance from the user. Face registration mode to collect images while rotating through.

When the robot rotates, the user may stare at the camera, which is an image capturing means of the robot, while rotating in the same direction in place, and an example of the image collected by the process is illustrated in FIG. 4.

In the case of Figure 2c is a face registration mode in which the robot rotates in place and the user rotates 360 degrees around the robot to collect images, a method that requires very active cooperation from the user. The face registration mode as shown in FIG. 2C has an advantage of collecting images reflecting various lighting changes because the user moves, and easier to collect images than the registration modes of FIGS. 2A and 2B because the robot rotates in place. Do. An example of the image collected in the mode of FIG. 2C is as shown in FIG. 5.

In the present invention, an appropriate mode may be selected from among the above-described face registration modes or a combination of a plurality of registration modes may be used. For example, an initial template with very high reliability can be created when all three modes are used.

Among the face registration modes as described above, the registration modes of FIGS. 2A and 2B except for FIG. 2C are performed through the registered image collecting unit 120, and the image registration process will be described with reference to FIG. 6.

6 is a flowchart illustrating an image registration process according to an exemplary embodiment of the present invention.

When the face registration mode is driven at the user's request, the robot moves slowly from the start position by the robot controller 128. While moving, the image input unit 100 obtains an image through a camera, which is an image capturing means mounted on the robot, and provides the image to the registered image collecting unit 120 (S600).

The face detector 122 of the registered image collector 120 performs face detection on the received image and then determines whether a face exists in the received image (S602 and S604).

Face detection in the present invention is implemented by an adaboosting technique.

As a result of the determination in S604, if no face is detected, the process proceeds to S600 and after receiving the next image from the image input unit 100, S602 and S604 are performed.

On the other hand, when the face is detected as a result of the determination in S604, the face detector 122 determines whether the face region is located at the center of the screen (S606).

As a result of the determination in S606, when the face area is not located at the center of the screen, the position controller 126 adjusts the position of the image capturing means mounted on the head of the robot so that the face area is located at the center of the screen ( S608), the process proceeds to S600 and the following steps are performed.

On the other hand, as a result of the determination in S606, the face detector 122 having the face region positioned in the center portion of the screen stores the corresponding image in the registered image collection buffer unit 135 (S610).

Thereafter, the moving robot measures the distance to the user in order to correct his moving trajectory in real time (S612). That is, the distance measuring unit 124 of the robot obtains distance measurement information by driving to a lower position (chest) of the face detected through a distance measuring sensor such as an ultrasonic wave or a laser sensor having a driving shaft such as a camera. The distance between the user and the robot is measured based on the distance measurement information. Here, an example of the robot equipped with the ultrasonic sensor for distance measurement, which is the distance measuring unit 124, is illustrated in FIG. 7.

Then, the robot controller 128 moves the robot while maintaining the movement trajectory of the robot based on the measured distance information (S614).

The robot controller 128 determines whether the robot has moved to the final position (S616), and when the determination result of S616 reaches the final position, generating the initial template of the user by driving the face information register 140. In S618, if not, the process proceeds to S600 to perform subsequent steps.

The face information registration unit 140 normalizes each image stored in the registered image collection buffer unit 135 and then detects each feature part, and generates an initial template of the user based on each detected feature part. ), The configuration includes a face normalizer 142, a feature extractor 144, and a template generator 146.

A process of generating an initial template by using each image stored in the registered image collection buffer unit 135 by the face information register 140 will be described with reference to FIG. 8.

8 is a flowchart illustrating an initial template generation process according to a preferred embodiment of the present invention.

Referring to FIG. 8, when each image stored in the registered image collection buffer unit 135 is input (S800), the face normalization unit 142 performs face detection by using an ad-boost method on the image and then detects the detected face image. In order to normalize, the position of the pupil is detected (S802 and S804).

The face normalizer 142 cuts a predetermined range based on both pupils after performing rotation correction and scaling using the detected position information of both pupils, thereby detecting the detected facial region. The pupil is aligned so that the pupil exists at the position (S806).

Then, the face normalization unit 142 verifies whether the normalized face image accurately includes the face region (S808). That is, since normalization of face images is performed based on both eyes, normalized face images do not correctly include face regions and fail verification when both eyes are incorrectly detected.

As a result of the verification of S808, when the verification passes, the face normalization unit 142 improves the image quality of the face area, that is, to remove the interference of the light after masking the acquired face image to remove the interference of the background. Histogram smoothing is performed on the face image portions other than the mask (S810).

Then, the feature extractor 144 extracts a multiple PCA vector by performing Principal Component Analysis (PCA) on the set face region, analyzes the edge distribution of the face image, and performs edge analysis. By extracting an edge distribution vector, a feature is extracted for a corresponding face region (S812).

When all the stored images of the registered image collecting buffer unit 135 are performed from S800 to S812, that is, when there are no more images in the registered image collecting buffer unit 135, the template generating unit 146 may apply to each image. The initial template is generated by combining the features of the face region and stored in the database 160 (S814 and S816).

Implementing the initial template generating device through the method proposed in the present invention, the result of 1370 times face recognition for a total of 10 people in the home environment. The recognition rate was 78.4% when the initial template was configured in the conventional camera fixed registration mode, and the excellent recognition rate of 92.8% was obtained when the initial template was configured in the registration mode of FIG. 2 (c) of the present invention. It has been confirmed that the performance of the robot's face recognizer can be improved.

It has been described so far limited to one embodiment of the present invention, it is obvious that the technology of the present invention can be easily modified by those skilled in the art. Such modified embodiments should be included in the technical spirit described in the claims of the present invention.

1 is a block diagram showing an initial template generating device in a robot according to an embodiment of the present invention,

2A is a view for explaining a mode in which a user looks at a robot in a fixed position and registers a user's face while the robot moves.

2B is a view for explaining a mode in which the user rotates in place and looks at the robot and the robot registers the user's face while rotating about the user.

FIG. 2 (c) is a view for explaining a mode in which the robot rotates in place and the user rotates around the robot to register a face of the user.

FIG. 3 is a diagram illustrating an example of a user face image acquired in the user registration mode of FIG. 2A.

4 is a diagram illustrating an example of a user face image acquired in the user registration mode of FIG. 2B.

FIG. 5 is a diagram illustrating an example of a user face image acquired in the user registration mode of FIG. 2C.

6 is a flowchart illustrating a process of operating a registered image collector of the present invention;

7 is a diagram illustrating an example of a robot equipped with an ultrasonic sensor for distance measurement.

8 is a flowchart illustrating a process of generating an initial template by operating the face information register of the present invention.

<Description of the code | symbol about the principal part of drawing>

100: image input unit 120: registered image collection unit

135: registration image acquisition buffer 140: face information registration unit

160: database

Claims (19)

An image input unit which acquires an image using an image photographing means mounted on a robot; A registered image collecting unit which rotates the robot and collects a registered image by receiving an image from the image input unit at a predetermined interval during the movement; The face information register unit for generating a user's initial template by detecting a feature part after normalizing the face area in the collected registered images. Initial template generation device comprising a. The method of claim 1, The registered image collecting unit, When the robot rotates 360 degrees around the user or the user rotates 360 degrees around the robot, the image at a predetermined interval when the user or the robot rotates in position according to the rotation direction of the robot or the user. Initial template generating device characterized in that the image is provided from the input unit. The method of claim 1, The registered image collecting unit, A face detector which receives an image from the image input unit and detects a face area of the user; A position controller which controls the position of the image capturing means based on the detected position on the screen of the face region Initial template generating device further comprising. The method of claim 3, wherein The initial template generating device, The apparatus may further include a registered image collection buffer configured to store an image in which the detected face region is located at a center position on a screen. And the face information registration unit generates an initial template by using each image stored in the registered image collection buffer unit. The method of claim 3, wherein The registered image collecting unit, A distance measuring unit which is driven based on the position of the face region and measures a distance between the robot and the user; Robot control unit for controlling the movement of the robot based on the distance measured by the distance measuring unit Initial template generating device further comprising. The method of claim 5, wherein The distance measuring unit, And an ultrasonic or laser sensor which is driven at an arbitrary position of the user to measure a distance between the user and the robot. The method of claim 1, The face information registration unit, A face normalizer which normalizes a face area in each image output from the registered image collector; A feature extractor which extracts a feature using each normalized face region; Initial template generation unit for generating an initial template by combining the extracted features Initial template generation device comprising a. The method of claim 7, wherein The face normalization unit, Means for detecting the position of the pupil in the face region; Means for cutting a predetermined range based on both pupils after performing tilt correction and size correction on the basis of the detected pupil positions; Means for verifying whether or not the normalized face image is correctly included in the face region; Means for performing image quality improvement on the verified face image Initial template generation device comprising a. The method of claim 8, Means for performing the image quality improvement, And a histogram smoothing on the face images other than the mask after applying the mask to the verified face image. The method of claim 7, wherein The feature extraction unit, Means for extracting multiple principal component analysis vectors by performing multiple principal component analysis on the normalized face region; Means for extracting an edge distribution vector by analyzing an edge distribution of a face image in the normalized face region; Means for extracting features using the edge distribution vector and multiple principal component analysis vectors Initial template generation device comprising a. (a) acquiring an image using image capturing means mounted to the robot at a predetermined interval during the movement of the robot while rotating the robot from an arbitrary position to a preset final position; (b) collecting a plurality of registered images by detecting a face region in the obtained image; (c) normalizing the face area of each registered image; (d) detecting feature portions in the normalized face region and generating an initial template of a user based on each detected feature portion; Initial template generation method comprising a. The method of claim 11, In step (a), When the robot is rotated 360 degrees around the user or the user is rotated 360 degrees around the robot, when the user or the robot rotates in place to the rotation direction of the robot or the user image from the imaging means Initial template generation method characterized in that the provided. The method of claim 11, In step (a), And receiving an image from the image capturing means at a predetermined interval when the robot moves while changing the rotation radius by a predetermined angle to the user center. The method of claim 11, In step (b), (b-1) determining whether a face region of the user is detected from the acquired image; (b-2) if the face region is not detected as a result of the determination, changing the position of the image capturing means and proceeding to step (b-1); (b-3) when the face region is detected as a result of the determination, storing the acquired image in the registered image collection buffer unit when the detected face region is in a preset position; Initial template creation method further comprising. The method of claim 14, In the step (b-3), when the detected face area is located at a center position on the screen of the acquired image, the initial template generation method of the registered image collection buffer unit is stored. The method of claim 14, In step (b), (b-4) measuring the distance between the user and the robot after storing the acquired image in the registered image collection buffer unit; (b-5) determining whether the robot has reached a predetermined final position after moving the robot based on the measured distance; As a result of the determination in (b-5), when the robot does not reach the final position, the (b-1), (b-2), (b-3), (b-4) and (b-5) Steps to perform Initial template creation method further comprising. The method of claim 11, In step (c), (c-1) detecting the position of the pupil in the face region of the registered image; (c-2) after performing a tilt correction and a size correction based on the detected pupil position, cutting out a predetermined range based on both pupils; (c-3) verifying whether the normalized face image is correctly included in the face region; (c-4) improving image quality of the verified face image Initial template creation method further comprising. The method of claim 17, Step (c-4) is, And applying a mask to the verified face image to perform histogram smoothing on the face images other than the mask to improve the image quality. The method of claim 11, In step (d), Performing a multi principal component analysis on the normalized face region to extract a multi principal component analysis vector; Extracting an edge distribution vector by analyzing an edge distribution of a face image in the normalized face region; Extracting a feature using the edge distribution vector and the multiple principal component analysis vector; Initial template creation method further comprising.

Images