CN113752983B - Vehicle unlocking control system and method based on face recognition/eye recognition - Google Patents
Vehicle unlocking control system and method based on face recognition/eye recognition Download PDFInfo
- Publication number
- CN113752983B CN113752983B CN202111094973.2A CN202111094973A CN113752983B CN 113752983 B CN113752983 B CN 113752983B CN 202111094973 A CN202111094973 A CN 202111094973A CN 113752983 B CN113752983 B CN 113752983B
- Authority
- CN
- China
- Prior art keywords
- face
- vehicle
- module
- recognition
- human
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R25/00—Fittings or systems for preventing or indicating unauthorised use or theft of vehicles
- B60R25/20—Means to switch the anti-theft system on or off
- B60R25/25—Means to switch the anti-theft system on or off using biometry
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/037—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for occupant comfort, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R25/00—Fittings or systems for preventing or indicating unauthorised use or theft of vehicles
- B60R25/20—Means to switch the anti-theft system on or off
- B60R25/25—Means to switch the anti-theft system on or off using biometry
- B60R25/255—Eye recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention relates to a vehicle unlocking control system and method based on face recognition/eye recognition, wherein the system comprises: the system comprises a low-power consumption awakening module, a communication module, an acquisition module, a biological identification module, a storage module, a control module and an alarm module, wherein the awakening module and the communication module are respectively connected with the acquisition module, the acquisition module is connected to the biological identification module, the biological identification module is connected to the storage module, the alarm module and the control module, and the control module is connected with the communication module; the control method comprises the following steps: the method comprises the following steps that firstly, interaction is carried out between a mobile phone and a vehicle control system, and a face image and data information are preset in the vehicle control system; secondly, carrying out face recognition, eye recognition and living body detection by using preset face images and data information; and step three, executing vehicle unlocking control after the face recognition or the eye recognition and the living body detection pass simultaneously.
Description
Technical Field
The invention relates to the technical field of vehicle control, in particular to a vehicle unlocking control system and method based on face recognition/human eye recognition.
Background
Currently, vehicles have become the main transportation tool for people to go out. At present, people must use a vehicle key when unlocking and controlling the vehicle. Sometimes, when people want to drive a car suddenly, the situation that the car key is not carried is likely to happen, and at the moment, people must find the car key or take other vehicles, so that much time is wasted. Moreover, the entity key is worn on the body when in use, so that the phenomenon of losing and forgetting can easily occur, and if the entity key is picked up by a lawbreaker, the property safety of the owner can be threatened.
The novel unlocking mode can be divided into a physical unlocking mode and a non-physical unlocking mode, wherein the physical unlocking mode comprises card swiping unlocking, NFC unlocking and the like, and the novel unlocking mode has the defect that unlocking cannot be completed when a physical key is not carried, and a car is required to be unlocked; non-physical unlocking modes comprise password unlocking, fingerprint unlocking and the like, and have the defects of being copied, easy leakage and no guarantee on safety. And after the automobile is unlocked, the automobile is not adjusted according to unlocking information, and after the automobile is unlocked, a driver needs to manually adjust the positions of a seat, a rearview mirror and the like, so that the automobile is very complicated.
Disclosure of Invention
In order to solve the problems, the face recognition/human eye recognition vehicle unlocking control system provided by the invention unlocks the vehicle by using technologies such as silence living body detection, face recognition, human eye recognition and the like, and provides personalized services for an unlocking person according to the identity of the unlocking person after unlocking, wherein the personalized services comprise automatic adjustment of the seat position, starting of an air conditioner, adjustment of the temperature favored by the unlocking person, playing of music favored by the unlocking person and the like. And in addition, a human eye recognition algorithm is used in the unlocking process, so that the mask and the scarf can be unlocked without being taken off.
The technical scheme of the invention is as follows: a face recognition/eye recognition vehicle unlock control system, comprising: the system comprises a low-power consumption awakening module, a communication module, an acquisition module, a biological identification module, a storage module, a control module and an alarm module, wherein the awakening module and the communication module are respectively connected with the acquisition module, the acquisition module is connected to the biological identification module, the biological identification module is connected to the storage module, the alarm module and the control module, and the control module is connected with the communication module; wherein:
the low-power consumption awakening module is configured to be a human body infrared induction module with micro-ampere standby power consumption or a button on a vehicle door handle without power supply, and other modules in the system are awakened when the human body infrared induction module induces a human body or someone presses the button.
The communication module is used for communicating with the mobile phone application, can be configured as equipment with a communication function, such as Bluetooth or a wireless network, and transmits the data in an encryption mode.
An acquisition module configured as a monocular camera, a binocular camera, a structured light camera, or a ToF camera. When the awakening module awakens the system, the acquisition module starts to acquire images for the biological identification module to use.
The biological recognition module is configured as a processor capable of rapidly operating a neural network to perform deep learning calculation, and can operate algorithms such as face detection, angle detection, definition detection, silence living body detection, face recognition, human eye recognition and the like.
The storage module is configured to a flash memory or a hard disk and other computer readable storage media, and is used for storing data required by a human face feature library, a human eye feature library, unlocking records and personalized services, the data comprises seat positions, air conditioner temperatures, song types and the like, and all the data are encrypted.
The control module is used for unlocking the car door, adjusting the position of the seat according to the personalized service data stored in the storage module, starting the air conditioner, adjusting the temperature and starting the sound box to play appropriate music.
The alarm module is configured as a sound-light alarm device, and when a stranger tries to unlock the vehicle for many times, the sound-light alarm device gives a sound-light alarm to the stranger.
According to another aspect of the present invention, a vehicle unlocking control method based on face recognition/eye recognition is provided, which includes the following steps:
the method comprises the following steps that firstly, interaction is carried out between a mobile phone and a vehicle control system, and a face image and data information are preset in the vehicle control system;
secondly, carrying out face recognition, eye recognition and living body detection by using preset face images and data information;
step three, executing vehicle unlocking control after the human face recognition or the human eye recognition and the living body detection pass simultaneously;
further, the first step is that the mobile phone is interacted with a vehicle control system, and a face image and data information are preset in the vehicle control system; the specific steps of presetting the face image and the data information are as follows:
1) When the awakening module detects that a person exists, awakening all other modules;
2) Connecting a vehicle unlocking control system through a communication module by using a mobile phone application;
3) The identity is verified on the mobile phone application, if the identity is not verified, the connection between the mobile phone and the vehicle unlocking control system is disconnected, otherwise, the following operation is carried out;
4) Inputting personnel information on a mobile phone application, transmitting the personnel information to a vehicle unlocking control system through a communication module, and simultaneously transmitting an instruction for starting acquisition;
5) The vehicle unlocking control system receives the acquisition instruction, and then the acquisition module starts to acquire images;
6) Detecting the face in the acquired image by using a deep learning face detection algorithm, if the face is detected, continuing the following step 7), and otherwise, acquiring the image again;
7) Detecting the face angle by using a face angle detection algorithm, judging whether the detected face angle and the definition meet the standard, if so, continuing the following step 8), and otherwise, acquiring the image again;
8) Extracting human face features by using a human face recognition algorithm, and extracting human eye features by using a human eye recognition algorithm;
9) Extracting the human face features and the human eye features, and encrypting and storing the extracted human face features and the human eye features and the corresponding personnel information transmitted in the mobile phone application in the step 3) into a storage module of the vehicle unlocking control system;
10 The collected human face pictures are transmitted to the mobile phone application through the communication module by the communication module, so that the user can confirm the use.
Further, the human face angle detection algorithm determines the posture of the human face by obtaining the angle information of the face orientation, and the posture is expressed by euler angles which are divided into three directions: the steps of calculating the face angle are as follows:
1) Firstly, defining a 3D face model with n key points, wherein n is defined according to tolerance of accuracy;
2) Detecting key points of a deep learning face to obtain key points of a 2D face corresponding to the 3D face;
3) Calculating a rotation vector according to the detected key points and the defined key points; the relationship between the image 2D points and the spatial 3D points is shown in equation (1):
wherein Z c Is the depth, i.e. the distance from the coordinate point to the camera, K is the internal reference matrix of the camera, (x, y) is the coordinates of the image midpoint, (Xw, yw, zw) is the coordinates of the corresponding 3-dimensional space, R is the rotation matrix, T is the translation matrix;
calculating a rotation matrix R and a translation matrix T by fitting according to the formula (1) through the n face key points detected in the step 2) and the coordinates of the n face key points corresponding to the 3-dimensional space;
4) Converting the rotation vector into an Euler angle, wherein the relation between the rotation vector and the Euler angle is shown as an equation (2):
and (3) calculating an Euler angle (psi phi gamma) by using the rotation matrix R calculated in the step (3) through a formula (2), wherein the psi phi gamma is respectively a roll angle, a yaw angle and a pitch angle.
Furthermore, the definition detection algorithm collects a plurality of face pictures, divides the pictures into different categories according to the definition of the faces, and trains the pictures by using a convolutional neural network to obtain an algorithm model for definition detection;
the face in the picture is cut out according to the face detection, the model is used for classifying, whether the face is clear or not is judged, the picture which is not clear is removed, the influence of the picture definition on silence living body detection, face recognition and human eye recognition is effectively reduced, and the unlocking time is saved.
Further, in the second step, preset face images and data information are used for face recognition or human eye recognition, and the specific steps are as follows:
1) When the awakening module detects that a person exists, awakening all other modules, and starting to acquire images by the acquisition module;
2) Detecting the face in the acquired image by using a deep learning face detection algorithm, if the face is detected, continuing the following steps, otherwise, acquiring the image again;
3) Judging whether the face angle and the definition detected in the step 2) meet the standard, if so, continuing the following step 4), and otherwise, acquiring the image again;
4) Judging whether the face detected in the step 2) is a living body by using a deep learning silence living body detection algorithm, if so, continuing the following steps, and otherwise, acquiring the image again; the deep learning silent in vivo detection algorithm detects real people as one class, and detects non-real people as one class, wherein a convolutional neural network is used for classification;
5) Judging whether the face detected in the step 2) is covered by a mask or a scarf by using a deep learning mouth covering detection algorithm, if the face is not covered by the mask or the scarf, extracting face features by using a face recognition algorithm, and if not, extracting eye features by using an eye recognition algorithm; the method comprises the steps of deeply learning a mouth shielding detection algorithm, classifying face which shields a mouth into one class, classifying face which does not shield the mouth into one class, and classifying the face by using a convolutional neural network;
6) And (3) comparing the face features or the eye features extracted in the step 5) with a face library stored in a system in a comparison mode of calculating the similarity of the two features, taking the feature with the highest similarity as a comparison result, comparing the similarity with a threshold, if the similarity is greater than the threshold, identifying the feature, and otherwise, judging the feature as a stranger. The similarity calculation is calculated using the cosine distance, and the threshold is set to 0.7.
Furthermore, the low-power consumption awakening module comprises multiple modes, if the low-power consumption awakening module is an infrared human body sensing module, a person can be detected when approaching, and whether the door needs to be opened or not is judged by detecting the reflected time; alternatively, the first and second electrodes may be,
if the low-power consumption awakening module is a button, when a person presses the button, the vehicle door needs to be opened.
Further, after the step three, the face recognition or the eye recognition and the living body detection pass simultaneously, the unlocking control of the vehicle is executed; the method comprises the following specific steps:
and comparing the extracted human face features or human eye features with the corresponding human face feature library or human eye feature library, and unlocking the vehicle and opening the vehicle door to start the vehicle if the comparison is passed. If the comparison fails, the image is collected again if the image is identified as a stranger, if the image is continuously identified as a stranger for multiple times and exceeds the system limit, the alarm module starts to give an alarm, records the image trying to be unlocked for the car owner to check, performs personalized service according to the unlocking identity, and automatically adjusts the seat position, adjusts the air conditioner temperature and plays music;
when the vehicle is driven, after the personnel adjust the vehicle, the adjusted information is recorded into the vehicle unlocking control system and is bound with the unlocking personnel for the use of the unlocked personalized service.
According to another aspect of the present invention, a method for controlling emergency unlocking of a vehicle is provided, in which unlocking is performed by a mobile phone in an emergency state or an initial state, but is only performed by a vehicle owner or an administrator, and the method includes the following specific steps:
1) When the awakening module detects that a person exists, awakening all other modules;
2) Connecting a vehicle unlocking control system through a communication module by using a mobile phone application;
3) The identity is verified on the mobile phone application, if the identity is not verified, the connection between the mobile phone and the vehicle unlocking control system is disconnected, and the following operations cannot be carried out; otherwise, executing the following operation;
4) The unlocking instruction is transmitted through the communication module, and after the vehicle receives the unlocking instruction, the vehicle is unlocked, the vehicle door is opened, and the vehicle can be started.
Has the advantages that:
compared with the prior art, the invention has the following advantages:
1. the door is unlocked by directly using face recognition or eye recognition without a key.
2. The silent in-vivo detection algorithm is used for identifying the false faces such as photos, videos and masks, and the anti-counterfeiting method is safe, free of interaction such as nodding heads, blinking and the like, and fast and natural.
3. And the angle detection is used for removing the large-angle face, so that the situation that the silence living body of the large-angle face is not detected or cannot be identified is avoided, and the unlocking process is accelerated.
4. And the unclear face is removed by using the definition detection, so that the situation that the silent living body detection of the unclear face is not enough or cannot be identified is avoided, and the unlocking process is accelerated.
5. By using the human eye recognition algorithm, the identity of an unlocking person can be accurately recognized without taking off the mask and the scarf, and the vehicle can be unlocked.
6. The system is in a standby state by default, only the awakening module works all the time, and other modules are awakened to work, so that the electric quantity is saved, and the service time is prolonged.
7. The mobile phone application is connected with the vehicle unlocking control system, only a face acquisition instruction preset by a face is sent, and the acquired face picture is stored in a storage module of the vehicle unlocking control system, so that a screen and a keyboard do not need to be arranged on a vehicle, the integral attractiveness of the vehicle is not influenced, and the cost is saved.
8. After the vehicle is unlocked, personalized services including but not limited to seat position adjustment, air conditioner temperature adjustment and appropriate music playing can be performed according to the identity of the person unlocking the vehicle.
9. And the biological identification algorithm is processed in a local off-line mode, networking is not needed, and information safety is guaranteed.
10. All models such as an algorithm model, a human face photo, a human face feature library, a human eye feature library, personalized parameters and the like are stored in the vehicle, and data safety is guaranteed.
11. Communication and storage encryption are carried out, and data security is guaranteed.
Drawings
FIG. 1: the invention relates to a general block diagram of a vehicle unlocking control system;
FIG. 2 is a schematic diagram: the invention discloses a face presetting process;
FIG. 3: the invention relates to a human face angle detection algorithm;
FIG. 4: the invention utilizes the human face and the human eyes to detect the unlocking process;
FIG. 5 is a schematic view of: the invention also discloses a mobile phone unlocking process.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments, and all other embodiments obtained by a person skilled in the art based on the embodiments of the present invention belong to the protection scope of the present invention without creative efforts.
According to the embodiment of the invention, a vehicle unlocking control system based on face recognition or human eye recognition is provided, the vehicle is unlocked by using technologies such as silence living body detection, face recognition and human eye recognition, and after unlocking, personalized service is provided for an unlocking person according to the identity of the unlocking person. And the unlocking process uses a human eye recognition algorithm, and the unlocking can be carried out without taking off the mask and the scarf.
The structural block diagram of the vehicle unlocking control system based on the face recognition or the eye recognition is shown in fig. 1 and comprises a wake-up module, a communication module, an acquisition module, a biological recognition module, a storage module, a control module and an alarm module.
1. Low-power consumption awakens module up
The low-power consumption awakening module is configured to be a human body infrared induction module with micro-ampere standby power consumption or a button on a vehicle door handle without power supply, and other modules in the system are awakened when the human body infrared induction module induces a human body or someone presses the button.
2. Communication module
The communication module is used for communicating with the mobile phone application, can be configured as equipment with a communication function, such as Bluetooth or a wireless network, and transmits the data in an encryption mode.
3. Acquisition module
The acquisition module is configured as a monocular camera, a binocular camera, a structured light camera, or a ToF camera. Because the power consumption is great when the camera normally works, the acquisition module keeps a standby state, and when the awakening module is awakened, the acquisition module starts to acquire images.
4. Biological recognition module
The biological recognition module is configured as a processor capable of rapidly calculating a neural network, and can run algorithms such as face detection, angle detection, definition detection, silence living body detection, face recognition, human eye recognition and the like.
5. Memory module
The storage module is configured to a flash memory or a hard disk and other computer readable storage media, and is used for storing data required by a human face feature library, a human eye feature library, unlocking records and personalized services, including seat positions, air conditioner temperatures, song types and the like, and all the data are encrypted.
6. Control module
The control module is used for unlocking the car door, adjusting the position of the seat according to the personalized service data stored in the storage module, starting the air conditioner, adjusting the temperature and starting the sound box to play appropriate music.
7. Alarm module
The alarm module is configured as a sound-light alarm device and alarms strangers when the strangers try to unlock the vehicle for multiple times.
According to another aspect of the present invention, a vehicle unlocking control method based on face recognition or eye recognition is provided, which includes the following steps:
the method comprises the following steps that firstly, interaction is carried out between a mobile phone and a vehicle control system, and a face image and data information are preset in the vehicle control system;
secondly, carrying out face recognition, eye recognition and living body detection by using preset face images and data information;
step three, executing vehicle unlocking control after the human face recognition or the human eye recognition and the living body detection pass simultaneously;
specifically, the step one is to interact with a vehicle control system through a mobile phone, and preset a face image and data information in the vehicle control system; the face presetting process is shown in fig. 2. The method comprises the following specific steps:
1) When the awakening module detects that a person exists, awakening all other modules;
2) Connecting a vehicle unlocking control system through a communication module by using a mobile phone application;
3) The identity is verified on the application of the mobile phone, the verification mode is not limited to passwords, patterns, fingerprints, human faces and the like, if the verification is not passed, the connection between the mobile phone and the vehicle unlocking control system is disconnected, and the following operations cannot be carried out;
4) Inputting information such as personnel names, serial numbers and the like on the mobile phone application, transmitting the information to the vehicle unlocking control system through the communication module, and simultaneously transmitting an instruction for starting acquisition;
5) The vehicle unlocking control system receives the acquisition instruction, and then the acquisition module starts to acquire images;
6) Detecting the face in the acquired image by using a deep learning face detection algorithm, if the face is detected, continuing the following step 7), and otherwise, acquiring the image again;
7) Detecting a face angle by using a face angle detection algorithm, judging whether the detected face angle and the definition meet the standard, if so, continuing the following steps, otherwise, acquiring an image again;
according to one embodiment of the invention, the face angle detection algorithm mainly obtains the angle information of the face orientation, thereby determining the pose of the face. Which can be generally expressed in terms of euler angles. The euler angle is mainly divided into three directions: roll angle, yaw angle, pitch angle, as shown in fig. 3. Specifically, the step of calculating the face angle is as follows:
1) Firstly, defining a 3D face model with n key points, wherein n can be defined according to the tolerance degree of self-alignment accuracy;
2) Detecting key points of a deep learning face to obtain key points of a 2D face corresponding to the 3D face;
3) Calculating a rotation vector according to the detected key points and the defined key points;
the relationship between the image 2D points and the spatial 3D points is shown in equation (1):
wherein Z c Is the depth, i.e., the distance from the coordinate point to the camera, K is the camera's internal reference matrix, (x, y) are the coordinates of the image midpoint, (Xw)Yw, zw) are the coordinates of the point corresponding to the 3-dimensional space, R is the rotation matrix, and T is the translation matrix.
And (3) calculating a rotation matrix R and a translation matrix T by fitting according to the formula (1) through the n face key points detected in the step 2) and the coordinates of the n face key points corresponding to the 3-dimensional space.
4) The rotation vector is converted to an euler angle.
The relationship between the rotation vector and the Euler angle is shown in formula (2):
calculating an Euler angle (psi phi gamma) by using the rotation matrix R calculated in the step 3 through a formula (2), wherein the psi phi gamma is respectively a roll angle, a yaw angle and a pitch angle;
because the precision of silence living body detection, face recognition or human eye recognition is reduced when the angle is too large, after the euler angle of the face rotation is calculated, generally, when the roll angle is greater than 30 degrees and the yaw angle or pitch angle is greater than 15 degrees, the following silence living body detection and face recognition or human eye recognition are not performed, the face detection is performed again, the situation that the silence living body detection, the face recognition or the human eye recognition is not easily performed on the face through the silence living body detection, the face recognition or the human eye recognition is avoided, and the recognition time is saved.
According to one embodiment of the invention, the definition detection algorithm provided by the invention collects a large number of face pictures, divides the pictures into different categories according to the definition degree of the faces, and trains by using a convolutional neural network to obtain an algorithm model capable of performing definition detection.
The face in the picture is cut out according to the face detection, the model is used for classifying, whether the face is clear or not is judged, the picture which is not clear is removed, the influence of the picture definition on silence living body detection, face recognition and human eye recognition can be effectively reduced, and the unlocking time is saved.
8) Extracting human face features by using a human face recognition algorithm, and extracting human eye features by using a human eye recognition algorithm;
9) Extracting the human face features and the human eye features, and encrypting and storing the extracted human face features and the human eye features and the corresponding personnel information transmitted in the mobile phone application in the step 3 into a storage module of the vehicle unlocking control system;
10 The collected face picture is transmitted to the mobile phone application through the communication module for the user to confirm.
And step two, using the preset face image and data information to perform face recognition or human eye recognition, as shown in fig. 4, the specific steps are as follows:
1) When the awakening module detects that a person exists, all other modules are awakened, and the acquisition module starts to acquire images. (according to the embodiment of the present invention, the wake-up module can be implemented in various ways, for example:
1. if the awakening module is an infrared human body sensing module with standby power consumption microampere, the awakening module can detect when a person approaches (the principle is that the infrared human body sensing module emits invisible infrared light, whether the person approaches is judged by receiving reflected infrared light), and whether the door needs to be opened is judged by detecting the reflected time;
2. if the awakening module is a button, when a person presses the button, the vehicle door needs to be opened.
The camera mounted position can be door B post, and light filling lamp or camera naked eye are visible, through observing its position, can guarantee that the people's face just to light filling lamp or camera, can aim at the people's face.
2) Detecting the face in the acquired image by using a deep learning face detection algorithm, if the face is detected, continuing the following steps, otherwise, acquiring the image again;
3) Judging whether the face angle and the definition detected in the step 2 meet the standard, if so, continuing the following step 4, otherwise, acquiring the image again;
4) And (3) judging whether the face detected in the step (2) is a living body or not by using a deep learning silence living body detection algorithm, if so, continuing the following steps, and otherwise, acquiring the image again. The deep learning silent in vivo detection algorithm detects real persons as one class, detects non-real persons as one class, and uses a convolutional neural network for classification;
5) And (3) judging whether the face detected in the step (2) wears a mask, a scarf and the like by using a deep learning mouth shielding detection algorithm, if not, extracting the face features by using a face recognition algorithm, and otherwise, extracting the eye features by using an eye recognition algorithm. Deep learning mouth occlusion detection algorithm, classifying the face with the occluded mouth as a class, and classifying the face without the occluded mouth as a class by using a convolution neural network;
6) And (3) comparing the face features or the eye features extracted in the step 5) with a face library stored in a system in a comparison mode of calculating the similarity of the two features, taking the feature with the highest similarity as a comparison result, comparing the similarity with a threshold, if the similarity is greater than the threshold, identifying the feature, and otherwise, judging the feature as a stranger. The similarity calculation was calculated using cosine distances, with a threshold set at 0.7.
After the human face recognition or the human eye recognition and the living body recognition pass simultaneously, performing unlocking control on the vehicle; the method comprises the following specific steps:
and comparing the extracted human face features or human eye features with the corresponding human face feature library or human eye feature library, and unlocking the vehicle and opening the vehicle door to start the vehicle if the comparison is passed. If the comparison is failed, the vehicle is identified as a stranger, the image is collected again, if the vehicle is identified as the stranger for a plurality of times continuously and exceeds the system limit, the alarm module starts to give an alarm, and the image which is attempted to be unlocked is recorded for the vehicle owner to check. Further, according to the unlocking identity, personalized service is carried out, the seat position is automatically adjusted, the air conditioner temperature is adjusted, music is played, and the like.
When the vehicle is driven, after the personnel adjust the vehicle, the adjusted information is recorded into the vehicle unlocking control system and is bound with the unlocking personnel for the use of the unlocked personalized service.
According to another embodiment of the present invention, a vehicle emergency contact control method is provided, in an emergency state or an initial state, a mobile phone can be used for unlocking, but only operated by a vehicle owner or an administrator, and a mobile phone unlocking process is shown in fig. 5. The method comprises the following specific steps:
1) When the awakening module detects that a person exists, awakening all other modules;
2) Connecting a vehicle unlocking control system through a communication module by using a mobile phone application;
3) The identity is verified on the application of the mobile phone, the verification mode is not limited to passwords, patterns, fingerprints, human faces and the like, if the verification is not passed, the connection between the mobile phone and the vehicle unlocking control system is disconnected, and the following operation cannot be carried out;
4) Transmitting an unlocking instruction through a communication module;
5) And after the vehicle receives the unlocking instruction, unlocking the vehicle, opening the vehicle door and enabling the vehicle to start.
Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, but various changes may be apparent to those skilled in the art, and it is intended that all inventive concepts utilizing the inventive concepts set forth herein be protected without departing from the spirit and scope of the present invention as defined and limited by the appended claims.
Claims (5)
1. A vehicle unlocking control method based on face recognition/human eye recognition is characterized by comprising the following steps:
the method comprises the following steps that firstly, interaction is carried out between a mobile phone and a vehicle control system, and a face image and data information are preset in the vehicle control system; the specific steps of presetting the face image and the data information are as follows:
1) When the awakening module detects that a person exists, awakening all other modules;
2) Connecting a vehicle unlocking control system through a communication module by using a mobile phone application;
3) The identity is verified on the mobile phone application, if the identity is not verified, the connection between the mobile phone and the vehicle unlocking control system is disconnected, and otherwise, the following operation is carried out;
4) Inputting personnel information on a mobile phone application, transmitting the personnel information to a vehicle unlocking control system through a communication module, and simultaneously transmitting an instruction for starting acquisition;
5) The vehicle unlocking control system receives the acquisition instruction, and then the acquisition module starts to acquire images;
6) Detecting the face in the acquired image by using a deep learning face detection algorithm, if the face is detected, continuing the following step 7), and otherwise, acquiring the image again;
7) Detecting the face angle by using a face angle detection algorithm, judging whether the detected face angle and the definition meet the standard, if so, continuing the following step 8), and otherwise, acquiring the image again;
8) Extracting human face features by using a human face recognition algorithm, and extracting human eye features by using a human eye recognition algorithm;
9) Extracting the human face features and the human eye features, and encrypting and storing the extracted human face features and the human eye features and the corresponding personnel information transmitted in the mobile phone application in the step 4) into a storage module of the vehicle unlocking control system;
10 The collected face picture is transmitted to the mobile phone application through the communication module for the user to confirm;
secondly, carrying out face recognition, eye recognition and living body detection by using preset face images and data information;
the human face angle detection algorithm determines the posture of the human face by obtaining the angle information of the face orientation, and the posture is expressed by Euler angles which are divided into three directions: the steps of calculating the face angle comprise the following steps:
1) Firstly, defining a 3D face model with n key points, wherein n is defined according to tolerance of accuracy;
2) Detecting key points of a deep learning face to obtain key points of a 2D face corresponding to the 3D face;
3) Calculating a rotation vector according to the detected key points and the defined key points; the relationship between the image 2D points and the spatial 3D points is shown in equation (1):
WhereinZ c Is the depth, i.e. the distance from the coordinate point to the camera, K is the internal reference matrix of the camera, (x, y) is the coordinates of the image midpoint, (Xw, yw, zw) is the coordinates of the corresponding 3-dimensional space, R is the rotation matrix, T is the translation matrix;
calculating a rotation matrix R and a translation matrix T by fitting according to the formula (1) through the n face key points detected in the step 2) and the coordinates of the n face key points corresponding to the 3-dimensional space;
4) Converting the rotation vector into an Euler angle, wherein the relation between the rotation vector and the Euler angle is shown as an equation (2):
Calculating the Euler angle using the rotation matrix R calculated in step 3 by equation (2)
，
The three angles are respectively a rolling angle, a yaw angle and a pitch angle;
and step three, executing vehicle unlocking control after the face recognition or the eye recognition and the living body detection pass simultaneously.
2. The vehicle unlocking control method based on face recognition/eye recognition as claimed in claim 1, wherein the determination of whether the detected face angle and sharpness meet the standard is performed by the following steps: collecting a plurality of face pictures, dividing the pictures into different categories according to the face definition degree, and training by using a convolutional neural network to obtain an algorithm model for definition detection;
the face in the picture is cut out according to the face detection, the model is used for classifying, whether the face is clear or not is judged, the picture which is not clear is removed, the influence of the picture definition on silence living body detection, face recognition and human eye recognition is effectively reduced, and the unlocking time is saved.
3. The vehicle unlocking control method based on face recognition/eye recognition as claimed in claim 1, wherein in the second step, face recognition, eye recognition and living body detection are performed by using preset face images and data information, and the specific steps are as follows:
1) When the awakening module detects that a person exists, awakening all other modules, and starting to acquire images by the acquisition module;
2) Detecting the face in the acquired image by using a deep learning face detection algorithm, if the face is detected, continuing the following steps, otherwise, acquiring the image again;
3) Judging whether the face angle and the definition detected in the step 2) meet the standard, if so, continuing the following step 4), and otherwise, acquiring the image again;
4) Judging whether the face detected in the step 2) is a living body by using a deep learning silence living body detection algorithm, if so, continuing the following steps, otherwise, re-collecting the image; the deep learning silent in vivo detection algorithm detects real persons as one class, detects non-real persons as one class, and uses a convolutional neural network for classification;
5) Judging whether the face detected in the step 2) is covered by a mask or a scarf by using a deep learning mouth covering detection algorithm, if not, extracting the face features by using a face recognition algorithm, otherwise, extracting the eye features by using an eye recognition algorithm; the method comprises the steps of deeply learning a mouth shielding detection algorithm, classifying face which shields a mouth into one class, classifying face which does not shield the mouth into one class, and classifying the face by using a convolutional neural network;
6) And (4) comparing the face features or the eye features extracted in the step 5) with a face library stored in a system in a comparison mode of calculating the similarity of the two features, taking the feature with the highest similarity as a comparison result, comparing the similarity with a threshold, if the similarity is greater than the threshold, identifying the feature, otherwise, judging the feature as a stranger, calculating the similarity by using a cosine distance, and setting the threshold to be 0.7.
4. The method as claimed in claim 1, wherein the wake-up module comprises a plurality of modes,
if the awakening module is an infrared human body induction module, a person can be detected when approaching, and whether the door needs to be opened or not is judged by detecting the reflection time; or if the awakening module is a button, if a person presses the button, the vehicle door needs to be opened.
5. The vehicle unlocking control method based on face recognition/eye recognition according to claim 1, wherein after the third step, face recognition or eye recognition and living body detection are passed simultaneously, vehicle unlocking control is executed; the method comprises the following specific steps:
the extracted human face features or human eye features are compared with a corresponding human face feature library or human eye feature library, if the extracted human face features or human eye features pass the comparison, the vehicle is unlocked, a vehicle door is opened, the vehicle can be started, if the extracted human face features or human eye features do not pass the comparison, the vehicle is identified as a stranger, images are collected again, if the vehicle is identified as the stranger for multiple times, system limitation is exceeded, an alarm module starts to give an alarm, the image trying to be unlocked is recorded for a vehicle owner to check, personalized service is carried out according to the unlocking identity, the seat position is automatically adjusted, the air conditioner temperature is adjusted, and music is played;
when the vehicle is driven, after the personnel adjust the vehicle, the adjusted information is recorded into the vehicle unlocking control system and is bound with the unlocking personnel for the use of the unlocked personalized service.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111094973.2A CN113752983B (en) | 2021-09-17 | 2021-09-17 | Vehicle unlocking control system and method based on face recognition/eye recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111094973.2A CN113752983B (en) | 2021-09-17 | 2021-09-17 | Vehicle unlocking control system and method based on face recognition/eye recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113752983A CN113752983A (en) | 2021-12-07 |
| CN113752983B true CN113752983B (en) | 2022-11-22 |
Family
ID=78796375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111094973.2A Active CN113752983B (en) | 2021-09-17 | 2021-09-17 | Vehicle unlocking control system and method based on face recognition/eye recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113752983B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114495328B (en) * | 2022-01-19 | 2024-02-20 | 深圳市凯迪仕智能科技股份有限公司 | Door lock device and control method thereof |
| CN115079164A (en) * | 2022-06-10 | 2022-09-20 | 中国第一汽车股份有限公司 | Vehicle exterior detection system, method, vehicle and storage medium |
| CN115393997A (en) * | 2022-08-31 | 2022-11-25 | 深圳立欧实业有限公司 | Automobile electronic unlocking method and device based on multi-dimensional data identification and electronic equipment |
| JP7413481B1 (en) | 2022-10-21 | 2024-01-15 | レノボ・シンガポール・プライベート・リミテッド | Information processing device and control method |
| CN116805430B (en) * | 2022-12-12 | 2024-01-02 | 安徽国防科技职业学院 | Digital image safety processing system based on big data |
| CN116061878A (en) * | 2022-12-20 | 2023-05-05 | 上海势航网络科技有限公司 | Vehicle keyless entry and starting method, device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104709238A (en) * | 2015-03-13 | 2015-06-17 | 广东欧珀移动通信有限公司 | Vehicle remote control method and mobile device |
| CN104786991A (en) * | 2015-04-21 | 2015-07-22 | 北京汽车股份有限公司 | Car control method and system based on fingerprint identification |
| JP2019061495A (en) * | 2017-09-26 | 2019-04-18 | 株式会社エクォス・リサーチ | Information processing apparatus |
| CN208861343U (en) * | 2018-08-29 | 2019-05-14 | 上海交通大学 | A kind of automobile system for unlocking based on recognition of face |
| CN211001243U (en) * | 2019-11-05 | 2020-07-14 | 上海锐镁新能源科技有限公司 | Keyless entry system based on face recognition |
| CN112036238A (en) * | 2020-07-24 | 2020-12-04 | 支付宝实验室(新加坡)有限公司 | Face data processing method and device, electronic equipment and storage medium |
-
2021
- 2021-09-17 CN CN202111094973.2A patent/CN113752983B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104709238A (en) * | 2015-03-13 | 2015-06-17 | 广东欧珀移动通信有限公司 | Vehicle remote control method and mobile device |
| CN104786991A (en) * | 2015-04-21 | 2015-07-22 | 北京汽车股份有限公司 | Car control method and system based on fingerprint identification |
| JP2019061495A (en) * | 2017-09-26 | 2019-04-18 | 株式会社エクォス・リサーチ | Information processing apparatus |
| CN208861343U (en) * | 2018-08-29 | 2019-05-14 | 上海交通大学 | A kind of automobile system for unlocking based on recognition of face |
| CN211001243U (en) * | 2019-11-05 | 2020-07-14 | 上海锐镁新能源科技有限公司 | Keyless entry system based on face recognition |
| CN112036238A (en) * | 2020-07-24 | 2020-12-04 | 支付宝实验室(新加坡)有限公司 | Face data processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113752983A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113752983B (en) | Vehicle unlocking control system and method based on face recognition/eye recognition | |
| JP7145887B2 (en) | Vehicle, vehicle door unlocking control method, device, and vehicle door unlocking system | |
| US10885396B2 (en) | Generating composite images using audio/video recording and communication devices | |
| US10679443B2 (en) | System and method for controlling access to a building with facial recognition | |
| US20180247504A1 (en) | Identification of suspicious persons using audio/video recording and communication devices | |
| CN109343698A (en) | Data processing system, computer implemented method and non-transitory computer-readable medium | |
| JP2003138817A (en) | Keyless entry system for vehicle, person identifying method, and person identifying system | |
| US20050055582A1 (en) | System and method for dynamic stand-off biometric verification | |
| WO2016019590A1 (en) | Intelligent door lock system, and intelligent door lock and intelligent alarm door | |
| US11115629B1 (en) | Confirming package delivery using audio/video recording and communication devices | |
| EP1955290B1 (en) | Distributed stand-off id verification compatible with multiple face recognition systems (frs) | |
| US20060082439A1 (en) | Distributed stand-off ID verification compatible with multiple face recognition systems (FRS) | |
| US10949692B2 (en) | 3D dynamic structure estimation using synchronized images | |
| JP4760049B2 (en) | Face authentication device, face authentication method, electronic device incorporating the face authentication device, and recording medium recording the face authentication program | |
| WO2019050766A1 (en) | Vein matching for difficult biometric authentication cases | |
| US11064167B2 (en) | Input functionality for audio/video recording and communication doorbells | |
| US20200013266A1 (en) | Adjustable alert tones and operational modes for audio/video recording and communication devices based upon user location | |
| CN115830670A (en) | Electronic device with face recognition function | |
| US10930126B1 (en) | Motion sensing for electronic devices | |
| KR102265624B1 (en) | Start-up security system for vehicles using facial recognition | |
| CN108550210A (en) | Control vehicle unlocks the method and apparatus of state | |
| CN108629278B (en) | System and method for realizing information safety display based on depth camera | |
| JP2003233816A (en) | Access control system | |
| US11390249B2 (en) | Vehicle vision system | |
| US11627289B1 (en) | Activating security system alarms based on data generated by audio/video recording and communication devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |