KR20170088315A - Integrated Control method for Vehicle Drive using Depth Information based Face Recognition - Google Patents

Abstract

The present invention relates to a vehicle control method for face recognition based on depth information and, more specifically, to a vehicle control technique which recognizes a face of a vehicle driver by a vehicle control system for face recognition having an algorithm and an apparatus therein to use depth information to perform face recognition, regards as a similar situation when a change in a depth value for a normal face feature of the driver recognized by the vehicle control system for face recognition occurs to propagate a warning or automatically set a vehicle environment such as a seat position suitable for the recognized driver, an angle of a room mirror and a side mirror, preferred music play, and a preferred radio frequency. According to the present invention, the vehicle control system for face recognition based on depth information comprises: a vehicle environment setting unit to set a vehicle driving environment; a driving-while-drowsy sensing unit having an algorithm therein to determine a similar situation when a change in a depth value for a normal face feature of a previously recognized driver occurs; and a depth information-based face recognition unit.

Description

[0001] The present invention relates to a depth information-based face recognition vehicle control method,

The present invention relates to a depth information-based facial recognition automobile control method, and more particularly, to a depth information-based face recognition automobile control method for recognizing a face of a driver using a facial recognition automobile control system including an apparatus for performing facial recognition using depth information, Facial Recognition Automatically recognizes the driver's normal facial features recognized by the facial recognition automobile control system as a similar situation when a change occurs in depth, propagates the alarm or sets seat position, room mirror and side mirror angle suitable for recognized driver, And a preference radio frequency setting, and the like.

Face Recognition technology is one of the fields of biometric recognition (Boimetrics), where the machine automatically identifies and authenticates people using unique feature information contained in each face. The facial image, which is relatively easy and naturally input from various image media, is used to separate faces from complex backgrounds and then search for positions such as eyes, nose, and mouth, sort and normalize size, extract feature information necessary for recognition, It is used for registration, recognition and authentication of faces by storing the template in a database by statistical methods. In the field of biometric systems, fingerprint, vein, and iris recognition fields are widely used and commercialized and commercialized. However, these biometrics systems have the disadvantage that human beings must be in touch with human beings, data collection is difficult, and intuitive. In this respect, facial recognition technology has the advantage that it is a non-contact type that takes an image and that the data recorded and used here is very intuitive in terms of face photographs.

This facial recognition technology is commonly used to compare patterns with previously stored reference images using image processing techniques. This is a typical application and example of using the optical inspection system in the printed circuit board (PCB) manufacturing line to determine the presence or absence of the PCB defect, the automatic license plate recognition for the intelligent traffic system, and the pattern matching technology in the object Internet technology. In order to perform such pattern recognition, when a conventional camera captures an image, a geometrically distorted image due to the perspective of the image depending on the camera position is photographed, which prevents the detection and recognition of the pattern from being performed smoothly. In order to correct the geometric distortion image due to the perspective, there has been proposed a distortion coefficient calculation method using a corrected object, a multi-directional pattern image using technique, and a geometric feature utilization technique of a line or a vanishing point. However, There is a problem that it can not be done. Also, there is a problem that the method of acquiring image information through color information is weak in an environment that affects color images.

In the case of imaging using a camera, the HEVC standard has recently been completed as a video encoding standard. The motion estimation and compensation method in video coding is essential for eliminating redundancy in the temporal direction. In H.264, HEVC, and so on, a block matching method is used to find a block that is a bundle of neighboring pixels in the spatial direction of the current screen and a closest block in the neighboring reference picture in the time direction for motion estimation. In the block matching motion estimation process, the evaluation scale for finding the block, the size of the search area in the reference screen, and the size of the current block should be considered. Since motion estimation accounts for more than 70% of implementation complexity in video coding, fast motion estimation methods that reduce complexity from the beginning of video coding have been studied. In this block matching motion estimation, the accuracy of estimation is high in the left and right movement of the camera and the spatial movement of the object in the image, but the accuracy of the image expansion and contraction is low. The expansion / contraction motion of the image means that the current screen is enlarged or reduced on the reference screen. In order to accurately extract the expansion / contraction ratio corresponding to enlargement / reduction, all possible expansion / contraction ratios must be applied. Furthermore, since the number of possible stretch ratios is too large, it is impossible to apply all cases. Therefore, techniques for reducing the complexity of implementation have been developed. In the first stage of the stretching motion estimation, a simple motion estimation is performed on several selected pixels. In the second step, a variety of iterative least-squares estimation is performed in consideration of a method of increasing the accuracy of all the pixels, calculation efficiency, A method of using an interpolated reference picture, and a method of simplifying motion estimation using a 3-D diamond search pattern have been proposed. However, since the conventional art uses only color information of the camera, it is difficult to estimate an accurate stretching motion vector, and there is also a problem that there is a limit in reducing the complexity of estimation.

A similar prior art to disclose a depth information based face recognition automobile control system according to the present invention is Korean Patent Registration No. 10-0778059 entitled " Drowsiness driving prevention device using face recognition technology and drowsiness driving prevention system using the same. The prior art includes a camera for photographing a driver's face; A user registration unit for storing and managing vector templates of the driver's face; A facial image acquiring unit for converting the analog facial image into a digital image stream and storing the digital facial image in a memory; A face image reading unit for generating a driver face vector template in the digital image stream; A drowsiness operation analyzing unit for comparing the driver face vector template with the pre-stored driver vector template to check the drowsiness state; And a drowsiness driving suppression unit for expressing the drowsiness suppression contents when the driving face of the driver is read from the drowsiness driving analysis unit to the drowsiness driving face.

Another prior art is the Korean Patent Registration No. 10-0961906 entitled " Method and Apparatus for Automatic Control of a Rearview Mirror Using Facial Recognition. &Quot; The similar prior art includes an image input step of photographing a driver's face reflected by a rearview mirror using a depth camera inside a car; An image distortion correction step of correcting the photographed image distortion; A lip position grasping step of grasping a driver's lip position in the corrected image; A driver space position grasping step of grasping a driver's space position through the grasped driver's lips position; And a rearview mirror correction angle calculation step of calculating a rearview mirror optimum angle for observing the vehicle rearview through the rearview mirror at the identified driver's location and calculating a rearview mirror correction angle as an initial angle difference of the rearview mirror, have.

Another similar prior art is Korean Patent Registration No. 10-1444537 entitled " Driver's face status notification device ". The similar prior art includes a camera (photographing means) for photographing the face of the driver, an image processing section (face state detecting section) for detecting the face state of the driver based on the face image of the driver photographed by the camera, And an image conversion unit (notification information creation means) that creates notification information that reflects the change of the face state based on the face state of the driver detected by the face detection unit A display for informing a person and a left and right lamp (notifying means).

However, in the related art prior art described above, the face depth value of a person is stored in advance in a database. When a person is photographed through a depth camera, the depth image of the person is corrected through the depth value, Extracting the feature points of the face from the stored database and the depth value of the captured image, comparing the depth value of the extracted face feature with the face data of the person stored in the database, and recognizing the depth value of the extracted face feature as the corresponding character It does not provide the implementation technology of the automobile control system driven by the depth information based face recognition algorithm.

KR 10-0778059 (B1) KR 10-0961906 (B1) KR 10-1444537 (B1)

The present invention aims to satisfy the technical needs required from the background of the above-mentioned invention. Specifically, it is an object of the present invention to provide a face recognition automobile control system having a depth image rotation module and an algorithm capable of correcting a phenomenon in which a perspective distortion of a face recognition image is generated in a conventional face recognition automobile control system There is a purpose. In order to solve the problem of difficulty in estimating the stretching motion vector in the conventional facial recognition automobile control system, there is a need to provide a technology for realizing a highly reliable facial recognition automobile control system by providing a face recognition automobile control system in which a depth information conversion module and an algorithm are embedded It has its purpose.

The technical objects to be achieved by the present invention are not limited to the above-mentioned problems, and other technical subjects not mentioned can be clearly understood by those skilled in the art from the following description. There will be.

To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described herein, a depth information-based face recognition automobile control system includes: a vehicle environment setting unit for setting a vehicle operation environment; A drowsy operation detecting unit having an algorithm for determining a similar situation when a depth value change of a recognized facial feature of a driver is detected; And a depth information-based face recognition unit.

As described above, the present invention has a depth information rotation module with improved processing speed and accuracy by correcting the perspective distortion generated in the corresponding color image by photographing one plane area through the depth camera and correcting the perspective distortion generated in the corresponding color image with depth information Therefore, even if a portrait is photographed on the side and not photographed on the front side, it is possible to calibrate the photographed person's own perspective, thereby enhancing the performance of the facial recognition automobile control system. In addition, since the depth image converting unit is provided to perform the correction to improve the accuracy of the comparison between the face currently photographed and the facial image stored in the authentication unit, the error according to the image capturing distance can be corrected by self- The performance of the system is increased.

It is to be understood that the technical advantages of the present invention are not limited to the technical effects mentioned above and that other technical effects not mentioned can be clearly understood by those skilled in the art from the description of the claims There will be.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of a main part of a depth-based face recognition automobile control system according to the present invention; FIG.
FIG. 2 is an exemplary view showing an installation position of a depth information-based face recognition automobile control system according to the present invention; FIG.
FIG. 3 illustrates an example of a portion of a depth information-based face recognition vehicle control system according to the present invention.
4 is a view illustrating an example of a depth image capturing using a depth image capturing unit of a depth information based face recognition automobile control system according to the present invention;
5 is a diagram illustrating a correction of an error generating pixel using a depth image correcting unit of a depth information-based facial recognition automobile control system according to the present invention;
6 is an illustration of interpolation of a spectacle frame of a spectacle wearer using a depth image correction unit of a depth information based face recognition automobile control system according to the present invention;
FIG. 7 is a block diagram of a detailed module of a depth image converting unit of a depth information-based face recognition automobile control system according to the present invention;
FIG. 8 is a flowchart illustrating a face-to-face alignment in a depth image converting unit of a depth-information-based face recognition automobile control system according to the present invention;
9 is a diagram illustrating a process of extracting a face using a depth value in a face detection unit of a depth information-based face recognition automobile control system according to the present invention;
10 is an exemplary view showing differences in depth values of main features extracted by the facial feature extraction unit of the depth information-based facial recognition automobile control system according to the present invention;
11 is an illustration of a facial region extracted by the facial feature extraction unit of the depth information-based facial recognition automobile control system according to the present invention;
FIG. 12 is an exemplary view showing a process of correcting the positions of two eyes in parallel to a horizontal line in order to correct the inclined state of the face; FIG.
FIG. 13 is a diagram illustrating a method of measuring a relative depth magnitude by calculating a depth difference between a nose and a nose in a face; FIG.
14 is an exemplary view for extracting jaw area using a depth information-based facial recognition automobile control system according to the present invention;
15 is an exemplary view of facial width measurement using a depth information-based facial recognition automobile control system according to the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, It is not. In the following description of the present embodiment, the same components are denoted by the same reference numerals and symbols, and further description thereof will be omitted.

Prior to the detailed description of each step of the invention, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary meanings, and the inventor shall design his own invention in the best manner It should be interpreted in the meaning and concept consistent with the technical idea of the present invention based on the principle that the concept of the term can be properly defined. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, It is to be understood that equivalents and modifications are possible.

1 to 3, the depth information-based facial recognition automobile control system according to the present invention includes a seat position, a room mirror, a side mirror angle setting, a preferred music reproduction, and a preferred radio frequency reproduction A vehicle environment setting unit (100) for performing an operation environment setting; A drowsy operation sensing unit 200 having an algorithm for determining a similar situation when a depth value change of a recognized facial feature of a driver is detected; And a depth information-based facial recognition unit (300).

The drowsiness operation sensing unit 200 may further include an alarm unit 210 for transmitting a danger alarm such as an alarm sound when the drowsy operation is sensed. , The closing of the eyelid, and the change in the depth value with respect to the mouth to be shed.

The face recognition unit 300 includes a face storage unit 310 for storing face feature depth information; Depth image capturing unit 320 for depth-of-field image capturing; A depth image correction unit 330 for depth value error correction; A facial image detector 340 for extracting a facial image portion; A depth image converting unit 350 for facial image stretching and face matching according to the image rotation / contraction transformation and the image capturing distance; A facial feature extraction unit 360 for extracting a depth image facial feature; A facial feature comparing unit 370 for comparing the data stored in the facial storing unit 310 with data stored in the facial storing unit 310; And a personality matching determination unit (380) for determining a person matching degree in a result of comparison with the stored data.

The face storage unit 310 stores depth information on the face of the driver to control the automobile so that the face recognition unit 300 recognizes the driver based on the depth information. The face storage unit 310 stores information on seating settings, preference music selection / playback information, seat position, room mirror, and side mirror angle for a plurality of recognized drivers. As shown in FIG. 2, the depth image radiographing section 320 is preferably located on a dashboard of a car. When the driver's face is photographed through the depth image radiographing section 320 to obtain a depth image, the facial feature extracting section 360 And the facial feature comparing unit 370. The facial feature comparing unit 370 may be a facial feature comparing unit. If the facial recognition unit 300 fails to recognize the driver's face, an alarm sound requesting re-photographing is firstly transmitted from the alarm alerting unit 210 and is not a driver previously stored in the face storage unit 310 If it is determined that the vehicle is stolen, the alarm alarm unit 210 transmits the alarm sound to the second alarm and sends a warning message to the predetermined contact.

In general, signs of drowsy driving are the driver's head nodding, the eyelid closing, yawning, the mouth opening larger than usual, and the movement of the body disappearing. Therefore, the depth image capturing unit 320 captures the above-described specific symptoms and can determine the drowsiness of the driver. When the drowsy operation sensing unit 200 detects any one of the above-mentioned indications, the alarm alarm unit 210 continuously transmits an alarm sound.

The facial features of the person to be identified are stored in the face storage unit 310 in a depth value form. In addition to the depth information of the face, the physical feature to be stored at this time is preferably the position and shape of the eyes, nose, and mouth, the width of the face, the height of the nose, and the outline information of the jaw. Also, the face storage unit 310 may be applied with a transformation using the depth image transform unit 350 so that the smallest depth value in the depth image has a reference depth value D _reference .

When the depth image radiographing unit 320 is installed at a position where a person can be photographed and a person is photographed, the face of the person is photographed as shown in FIG. At this time, since the nose portion is located nearest to the depth image radiographing unit 320 as a feature of the obtained depth image, the depth value is the smallest. Also, the depth values of the face and other regions are greatly different, so that the facial portion can be extracted using the face detector 340. A pixel in which an error occurs in the photographed image may be generated when the depth image is photographed and acquired through the depth image radiographing unit 320. [ At this time, interpolation is performed using the depth image correcting unit 330 as shown in FIG. 5 to correct the pixel where the error occurs.

In the facial recognition process, the wearer of the glasses may be subject to face recognition. In the case where the wearer of the eyeglass is a subject of depth image taking, the eyeglass lens is glass, so it does not affect the measurement of the depth value, but in the case of the eyeglass frame, it is measured that there is depth value, which may cause an error in the face recognition process. Therefore, the spectacle frame can be distinguished from the face area by using the fact that the characteristic of the spectacle frame is smaller than the average depth of the face. Also, by interpolating the area of the spectacle frame with the peripheral depth value as shown in FIG. 6, it is possible to acquire an image in which the spectacle area is removed from the depth image.

The process of extracting the face using the depth value in the face detector 340 is as follows. The person is photographed by the depth image photographing unit 320 and the area is separated through labeling according to the depth value. At this time, since the face is located close to the depth image radiographing unit 320, the average depth value of the face region is the smallest. This can be used to separate the face from other parts of the body, and a depth image that appears brighter as the depth value is lowered. In this case, the depth of the facial region is lower than that of the other region, so that the facial region can be separated from other regions.

Since the depth image radiographing section 320 may not always photograph the person in front of the radiographer, the depth image radiographing section 350 may apply the transform to align the face.

7, the depth image transform unit 350 includes a depth information calculator 330 for calculating depth information of an image of a plane photographed by the depth image radiographing unit 320, 341); A coordinate transforming unit 342 for calculating the position of each pixel in the coordinate system of the depth image radiographing unit 320 using the depth information calculated by the depth information calculating unit 341; A local normal vector calculating unit 343 for calculating a local normal vector of the pixel using the peripheral information of each pixel calculated by the coordinate transforming unit 342; A plane normal vector calculating unit 344 for obtaining a normal vector of the entire plane by using the local normal vector obtained by the local normal vector calculating unit 343; A transformation matrix calculation is performed to obtain a translation matrix by obtaining a translation matrix by calculating the rotation axis and an angle between the image and obtaining a translation matrix using the depth value of the image and the reference depth value in the face storage unit 310, A portion 345; And a transformation applying unit (346) applying transformation using the transformation matrix to align the facial image in the image so as to be comparable to the facial expression storage unit (310).

The coordinate transforming unit 342 photographs a plane through the depth image capturing unit 320 and obtains the position P (x, y) of the pixel in the depth image and the depth image capturing unit 320, (X, y) from the xy plane centered on the depth image radiographing unit 320 and using the factors of the depth image radiographing unit 320 and the D (x, y) And the depth optical image capturing unit 320 has a front optical axis direction as a z-axis.

Here, the transformation into the coordinate system of the depth image pickup unit 320 is performed by the position information P (x, y) of the pixel in the depth image coordinate system having the upper left corner of the image as the origin, y) is the position on the coordinate system which the image center is the origin where P (x, y) of the pixels in said depth image taking unit 320, the distance z axis in the coordinate system, z _c is the image coordinate system information P _v ( x _v , y _v ).

In order to calculate the position of each pixel in the coordinate system of the depth image radiographing section 320, the coordinate transforming section 342 transforms the image using the viewing angle and resolution, which are internal information of the depth image radiographing section 320 The distance f to the viewport is obtained by _obtaining the distance f to the viewport through the vertical viewing angle f _oVv of the depth image radiographing section 320 and the vertical resolution h of the depth image radiographing section 320 or by _{calculating the} horizontal viewing angle f _oVh and the depth imaging obtained by the horizontal resolution w of the unit 320, a position in the thus obtained wherein the depth of the image taking unit 320, using the distance f of the viewport to the coordinate system P _c (x, y) = (x _c, y _c , z _c ), and obtains the position information in the coordinate system of the depth image photographing unit 320.

If the position in the coordinate system of the depth image pickup unit 320 at the position P (x, y) in the image coordinate system is P _c (x, y), the local normal vector calculator 343 calculates location information in the depth image taking unit 320, the coordinate system of the point _{P c (x, y + 1} ), P c (x, y-1) and said of a point located on the left and right depth image taking unit 320 in the coordinate system the position information _{P c (x + 1, y} ), P c (x-1, y) two vectors _{v 1 = P c (x +} 1, y) on the basis of _{- P c (x-1,} y), in _{P c (x, y-1} ) produced and N _xy = the v ₁ × v pixel P (x, y) is obtained the cross product of two vectors with a _{2 - v 2 = P c (} x, y + 1) And a local normal vector N _xy is obtained.

The plane normal vector calculating unit 344 obtains the normal vector N in the plane region by adding the local normal vector of each pixel obtained by the local normal vector calculating unit 343, The image taken by the photographing unit 320 is transformed into a plane parallel to the z-axis by a normal vector N of the plane through rotation transformation, and the plane image is corrected to an image parallel to the xy plane to remove perspective distortion. And the unit vector of the plane normal vector after the rotation transformation is N '(0, 0, 1).

If the unit vector of the plane normal vector after the rotation transformation is N ', the unit vector which is the axis of the rotation transformation is u = (N x N' ) / (| N × N ' |) conversion vector _{u (u x, u y,} u z) to a normalized cross product of each of the normal vectors of the forward and backward through a rotation angle θ is θ = cos ^-1 ((N and N ') / (| N || N' |)). In the transformation matrix calculator 345, the rotation transformation matrix R

R = cos? I + sin? [U] + (1-cos?) U? U,

uⓧu =

을 통하여 구하고 검출된 안면 영역 내 깊이 값이 제일 작은 값을 D_min로 두고, 상기 안면 저장부(100)의 D_reference를 이용하여 평행이동 행렬 T₁, T₂를 [u] _x =

D _min and the parallel movement matrices T ₁ and T ₂ are calculated using the D _reference of the face storage unit 100

, T₂=

로 두어, 각 화소의 상기 깊이영상 촬영부(200) 좌표계에서의 P_c(i, j)를,T ₁ =

, T ₂ =

And P _c (i, j) in the coordinate system of the depth image capturing unit 200 of each pixel,

=R

를 통해 변환을 하여 변환 후의 위치 P'_c(i, j)=(x'_c, y'_c, z'_c)를 구하는 것을 특징으로 한다.

= R

By the conversion through characterized in that to obtain a position _{P 'c (i, j)} = (x' c, y 'c, z' c) after the conversion.

Assuming that distance f, the position after the obtained converted to a rotational transformation _{P 'c (i, j)} = (x' c, y 'c, z' c) of the conversion application unit 346 to the viewport in which an image is projected , P 'is x to convert them because the coordinates on the depth of the image taking unit 320, the coordinate system to image coordinates _{back' v = (x 'c f} ) / z' c, y 'v = (y' c f) / 'position in the image coordinate system by using the _c origin is present in the center of the screen P' z _v (x, _v, y, _v) is converted to the next set to the origin, as again the original pixel P (x, interp converted to y ) pixel P '(x' mapped to, y ') to obtain that, the pixel value of the depth value in the depth image P' (x ', y') to z 'to change to _c the face to face in the image And is arranged so as to be comparable to the storage unit (310).

Referring to FIG. 8, the operation of performing facial image alignment in the depth image transform unit 350 includes calculating depth information in an image of a plane photographed by the depth image photographing unit 320 (S341); Calculating (s342) a position of each pixel in the coordinate system of the depth image capturing unit 200 using the calculated depth information; Calculating (S343) a local normal vector of the pixel using the calculated peripheral information of each pixel; Obtaining a normal vector of the entire plane using a local normal vector (s344); Calculating a rotation axis and an angle between the image and obtaining a transformation matrix using the depth information of the center of the image and the reference depth value in the face storage unit (S345); And applying a transform using the transform matrix (s346).

The facial feature extraction unit 360 extracts features of the facial features in order to compare facial features stored in the facial feature storage unit 310 after the facial feature detection unit 340 detects the facial features. The feature of the face extracted here is preferably the face contour, the depth and position of the eye / nose / mouth /, the shape of the jaw, the height of the cheekbone, the height of the eyebrow bone, the height of the nose, and the face width. First extract the contours of the face and then extract the eyes / nose / mouth. The depth value of the face can be detected by using this feature because the nose region is the lowest and the eye region is relatively large. In addition, although the depth value of the mouth is larger than the depth value of the nose, since the mouth is protruded rather than the other facial parts, the depth value may be relatively small, so that feature extraction for eyes / nose / mouth is possible using this point 10; Fig. 11). Detects the detected eye / nose / mouth contour in the above process and detects the relative position of the eye / nose / mouth position. At this time, if the face is tilted, the eye / nose / mouth position fluctuation may occur, so that the position of the two eyes rotates the depth image parallel to the horizontal line. Then, the position of the two eyes, the position of the nose, and the position of the mouth are measured based on the center point of the positions of the two eyes (FIG. 12). Also, the height of the nose can be extracted, and the depth difference between the nose and the nose can be measured by measuring the nose and relative depth from the face through the depth image capturing unit 320 (FIG. 13). The height of the cheekbone below the eye and the height of the eyebrow bone above the eye are measured in the manner described above, and utilized as a feature of the face. After that, the shape of the jaw area is extracted (FIG. 14), and the jaw is regarded as the area from the area below the lip to the bottom of the face, thereby extracting the shape of the outline. The face width is measured by measuring the face width, the actual distance can be obtained through the depth value, the relative position in the depth image, and the internal factor of the depth image photographing unit 320, (FIG. 15).

If facial features that can specify a person are extracted, the facial feature comparing unit 370 performs a comparison operation on the feature data of the respective persons stored in the facial storing unit 310. If the comparison result is less than or equal to a certain degree of similarity, The personality coincidence judgment unit 380 judges that the person is not a specific person. Also, when the facial feature comparing unit 370 performs a comparison operation to confirm that all the features coincide with each other, the facial feature comparing unit 380 determines that the user is a specific person.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, will be. Accordingly, the true scope of the present invention should be determined only by the appended claims.

Claims (4)

A car environment setting unit (100) for performing a car driving environment setting; A drowsy operation sensing unit 200 having an algorithm for determining a similar situation when a depth value change of a recognized facial feature of a driver is detected; And a depth information-based facial recognition unit 300,
The face recognition unit 300 includes a face storage unit 310 for storing face feature depth information; Depth image capturing unit 320 for depth-of-field image capturing; A depth image correction unit 330 for depth value error correction; A facial image detector 340 for extracting a facial image portion; A depth image converting unit 350 for facial image stretching and face matching according to the image rotation / contraction transformation and the image capturing distance; A facial feature extraction unit 360 for extracting a depth image facial feature; A facial feature comparing unit 370 for comparing the data stored in the facial storing unit 310 with data stored in the facial storing unit 310; And a person match determining unit (380) for determining a person match degree in a result of comparison with the stored data,
The depth image converting unit 350 includes a depth information calculating unit 341 for calculating depth information on an image of a plane photographed by the depth image photographing unit 320; A coordinate transforming unit 342 for calculating the position of each pixel in the coordinate system of the depth image radiographing unit 320 using the depth information calculated by the depth information calculating unit 341; A local normal vector calculating unit 343 for calculating a local normal vector of the pixel using the peripheral information of each pixel calculated by the coordinate transforming unit 342; A plane normal vector calculating unit 344 for obtaining a normal vector of the entire plane by using the local normal vector obtained by the local normal vector calculating unit 343; A transformation matrix calculation is performed to obtain a translation matrix by obtaining a translation matrix by calculating the rotation axis and an angle between the image and obtaining a translation matrix using the depth value of the image and the reference depth value in the face storage unit 310, A portion 345; And a transformation applying unit (346) applying transformation using the transformation matrix to align the face in the image so as to be comparable to the face storage unit (310). The control method comprising:
A car environment setting step of setting a car driving environment;
Recognizing the driver's face using depth information of the image;
Determining a drowsy operation when a depth value of a previous image of the driver's face and a face image of the current driver are changed;
And an alarm alarm unit 210 for transmitting a danger alarm such as an alarm sound when it is determined that the drowsy driving operation is performed. The determining element for the drowsy operation includes an irregular movement of the face, an eyelid closure, And detects a change in the depth value of the object,
The step of recognizing the face
A step of photographing a depth image for depth-of-field imaging;
Correcting the depth value for the depth value error correction;
Detecting a face for extracting a depth image face portion;
Converting a depth image for facial image stretching and facial alignment according to image rotation stretching conversion and image capturing distance;
Extracting facial features for facial feature extraction;
Comparing facial features to be compared with data stored in the facial storage 310;
And judging whether or not the person matches with the stored data in order to determine the person's degree of consistency,
The step of transforming the depth image
Calculating depth information for calculating depth information on an image of a plane photographed in the depth image photographing step; A coordinate transforming step of calculating a position in the coordinate system of the depth image photographing unit 320 using the depth information calculated in the depth information calculating step;
Calculating a local normal vector for calculating a local normal vector of the pixel using the peripheral information of each pixel calculated in the coordinate transformation step;
A plane normal vector calculating step of obtaining a normal vector of the entire plane by using the local normal vector obtained in the local normal vector calculating step; Calculating a transformation matrix for calculating a rotation matrix by calculating an angle between the rotation axis and the depth of the depth image; Applying a transformation to apply a transform using the transform matrix,
When the facial recognition unit 300 fails to recognize the driver's face, an alarm sound requesting re-photographing is firstly transmitted from the alarm alerting unit 210 and is not a driver previously stored in the face storage unit 310 If it is determined that the vehicle is stolen, the alarm alarm unit 210 transmits the alarm sound to the second alarm alarm unit 210,
The spectacle frame can be distinguished from the face area by using the fact that the spectacle wearer has a depth value smaller than the average depth of the face in the case of the depth image taking subject, Wherein the depth information is obtained by removing an eyeglass area from the depth image. The method according to claim 1,
(X, y) in the depth image and a depth value D (x, y) in the depth P (x, y) by photographing a plane through the depth image photographing unit 320. [ And converts the depth image radiographing unit 320 into a coordinate system of the depth image radiographing unit 320 with the focus of the depth image radiographing unit 320 as the origin and the direction of the front optical axis of the depth image radiographing unit 320 as the z axis,
The local normal vector calculating unit 343 calculates the local normal vector of each of the depth information P _c (x, y + 1) and P _c (x, y-1) in the coordinate system of the depth image photographing unit 320, image taking unit 320, position information of the coordinate system _{P c (x + 1, y} ), P c (x-1, y) for, based on two vectors _{v 1 = P c (x +} 1, y) - P c (x-1, y), v 2 = P c (x, y + 1) - a _{P c (x, y-1} ) generate and N _xy = v cross product of two vectors by the relationship ₁ × v ₂ To obtain a local normal vector N _xy at P (x, y)
The plane normal vector calculating unit 344 obtains the normal vector N in the plane region by adding the local normal vector of each pixel obtained by the local normal vector calculating unit 343, The image captured by the photographing unit 320 is subjected to rotational transformation to make the normal vector N of the plane parallel to the z axis and the plane image to be parallel to the xy plane to remove the perspective distortion. A method for controlling a vehicle control system using depth information. The method according to claim 1,
In the depth image converting unit 350,
Calculating depth information from the image of the plane photographed by the depth image photographing unit 320 (s341);
Calculating (s342) a position of each pixel in the coordinate system of the depth image photographing unit 320 using the calculated depth information;
Calculating (S343) a local normal vector of the pixel using the calculated peripheral information of each pixel;
Obtaining a normal vector of the entire plane using a local normal vector (s344);
Calculating a rotation matrix by calculating an angle between the rotation axis of the image and the angle (s345);
And correcting the distortion of the image according to the position of the depth image radiographing section (320) by applying rotation transformation (S346). The method according to claim 1,
In the detecting the face, the person is photographed by the depth image photographing unit 320 in the face detection using the depth value, and the area is separated by labeling according to the depth value. The face is detected by the depth image photographing unit 320 Because of the difference in the average depth value due to the close proximity to the face,
The facial feature extraction unit 360 extracts facial features extracted from the facial features stored in the facial storage unit 310 after facial detection by the facial detection unit 340. The characteristics of the extracted facial features include a facial contour, The shape of the jaw, the height of the cheekbone, the height of the eyebrow bone, the height of the nose, and the face width,
The feature data of each person stored in the face storage unit 310 is compared in the facial feature comparison unit 370. If the comparison result is less than or equal to a certain degree of similarity, the personality matching determination unit 380 determines that the person is not a specific person And the facial feature comparing unit 370 performs a comparison operation. If it is determined that all features match, the personality determining unit 380 determines that the person is a specific person. Method of controlling the system.

Images