KR20220016184A - Vehicle door unlocking method and device, system, vehicle, electronic device and storage medium - Google Patents

Abstract

The present invention relates to a vehicle door unlocking method and apparatus, system, vehicle, electronic device and storage medium, the method comprising: searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on the vehicle (S11); In response to being able to search for the Bluetooth device of the predetermined identifier, establishing Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier (S12); Wake up and control an image acquisition module mounted on the vehicle to acquire a first image (S13), perform face recognition based on the first image (S14), and according to success of face recognition, and sending a door unlocking command and/or door opening command to one or more doors of the vehicle (S15), wherein the embodiment of the present invention meets demands such as operation at low power consumption, quick unlocking, and improvement of user experience. can satisfy

Description

Vehicle door unlocking method and device, system, vehicle, electronic device and storage medium

This application was filed with the State Intellectual Property Office of China on July 1, 2019, with an application number of 201910586845.6, and the title of the invention is "Method and apparatus, system, vehicle, electronic device and storage medium for unlocking a vehicle's door" Claims the priority of the patent application, all of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to vehicle technology, and more particularly, to a method and apparatus, system, vehicle, electronic device and storage medium for unlocking a vehicle door.

Vehicle unlocking by face authentication is a new technology for smart cars. Until now, in order to be able to detect faces in a timely manner, it is necessary to keep the camera on state. Needs to be. Thereby, the vehicle owner can be identified and unlocked quickly. However, in this method, since operation power consumption is high, the vehicle cannot be started due to lack of power due to operation of high power consumption for a long period of time, which may affect normal use of the vehicle and user experience.

The present invention provides a technical means for unlocking a door lock of a vehicle.

According to a first aspect of the present invention,

Searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle;

establishing Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier as the Bluetooth device of the predetermined identifier can be searched;

waking up and controlling the image acquisition module mounted on the vehicle to acquire a first image of the target object according to the success of the Bluetooth pairing/connection;

performing face recognition based on the first image;

A method of unlocking a door of a vehicle is provided, the method comprising sending a door unlocking command and/or a door open command to one or more doors of the vehicle upon success of facial recognition.

According to a second aspect of the present invention,

Searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle;

waking up and controlling an image acquisition module mounted on the vehicle to acquire a first image of a target object as the Bluetooth device of the predetermined identifier can be searched;

performing face recognition based on the first image;

A method of unlocking a door of a vehicle is provided, the method comprising sending a door unlocking command and/or a door open command to one or more doors of the vehicle upon success of facial recognition.

According to a third aspect of the present invention,

a search module for searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle;

As the Bluetooth device of the predetermined identifier can be searched for, a Bluetooth pairing/connection is established between the Bluetooth module and the Bluetooth device of the predetermined identifier, and a first image of the target object is acquired according to the success of the Bluetooth pairing/connection. Control by waking up the image acquisition module mounted on the vehicle to control it, or by waking up and controlling the image acquisition module mounted on the vehicle to acquire the first image of the target object according to the ability to search for a Bluetooth device with a predetermined identifier a wake-up module for

a face recognition module for performing face recognition based on the first image;

A door unlocking device for a vehicle is provided having an unlocking module for sending a door unlocking command and/or a door open command to one or more doors of the vehicle upon success of facial recognition.

According to a fourth aspect of the present invention, there is provided a memory, a face recognition module, an image acquisition module, and a Bluetooth module, wherein the face recognition module is respectively connected to the memory, the image acquisition module and the Bluetooth module, and the The Bluetooth module includes a microprocessor for waking up the face recognition module when Bluetooth pairing/connection with the Bluetooth device of a predetermined identifier is successful or when the Bluetooth device of the predetermined identifier can be searched; a communication interface for accessing the door domain controller is additionally installed in the face recognition module, and when face recognition is successful, control information for unlocking the door is transmitted to the door domain controller through the communication interface through the communication interface A transmission-mounted face authentication unlock system is provided.

According to a fifth aspect of the present invention, there is provided a vehicle comprising the vehicle-mounted face authentication unlock system, wherein the vehicle-mounted face authentication unlock system is connected to a door domain controller of the vehicle.

According to a sixth aspect of the present invention,

processor and

a memory for storing instructions executable by the processor;

There is provided an electronic device, wherein the processor is configured to execute the method according to the first aspect.

According to a seventh aspect of the present invention,

processor and

a memory for storing instructions executable by the processor;

There is provided an electronic device, wherein the processor is configured to execute the method according to the second aspect.

According to an eighth aspect of the present invention, there is provided a computer-readable storage medium storing computer program instructions, wherein the computer-readable storage medium realizes the method according to the first aspect when the computer program instructions are executed by a processor. provided

According to a ninth aspect of the present invention, there is provided a computer-readable storage medium storing computer program instructions, wherein the computer-readable storage medium realizes the method according to the second aspect when the computer program instructions are executed by a processor. provided

According to a tenth aspect of the present invention, a computer program including the computer readable code, the computer readable code executing instructions for realizing the method in a processor of the electronic device when the computer readable code operates in an electronic device program is provided.

In an embodiment of the present invention, as a Bluetooth device of a predetermined identifier can be searched for, Bluetooth pairing/connection is established between the Bluetooth module and a Bluetooth device of a predetermined identifier, and the face recognition module wakes up according to the success of Bluetooth pairing/connection. Thus, by controlling the image acquisition module to acquire the first image of the target object, the wakeup error probability of the facial recognition module can be effectively lowered by the method of waking up the face recognition module after successful Bluetooth pairing/connection, , it is possible to improve the user experience and effectively reduce the power consumption of the face recognition module. In addition, the Bluetooth pairing/connection method has advantages in that it provides high security and supports long-distance communication compared to short-range sensor technologies such as ultrasonic waves and infrared rays. According to practice, the time it takes for a user who has a Bluetooth device with a predetermined identifier to arrive at the vehicle through this distance (the distance between the user and the vehicle when Bluetooth pairing/connection is successful), and the vehicle sleeps the face recognition module It is shown that the time to wake up to transition from the state to the active state is approximately coincident. Accordingly, instead of waiting for the wakeup of the face recognition module after the user arrives at the door, when the user arrives at the door, unlocking by face recognition can be immediately performed by the wakeup face recognition module, and face recognition efficiency is improved. and improve the user experience. In addition, since the user does not feel Bluetooth pairing/connection, the user experience can be further improved. Therefore, in the embodiment of the present invention, the method of waking up the face recognition module after successful Bluetooth pairing/connection is a solution that can balance each aspect such as power saving, user experience and security of the face recognition module is providing

In addition, the above schematic description and the following detailed description are illustrative and interpretative only, and do not limit the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS Other features and aspects of the present invention will become apparent by describing exemplary embodiments in detail below with reference to the drawings.

Here, the drawings included as a part of the present specification are preferred for the embodiment of the present invention, and together with the specification are used for the description of the technical solution of the present invention.
1 is a flowchart illustrating a method for unlocking a door lock of a vehicle according to an embodiment of the present invention.
2 shows a schematic diagram of a B-pillar of a vehicle.
3 is a schematic diagram illustrating a mounting height of a vehicle door unlocking device and an identifiable height range in the vehicle door unlocking method according to an embodiment of the present invention.
4A is a schematic diagram of an image sensor and a depth sensor in a method for unlocking a door of a vehicle according to an embodiment of the present invention.
4B is another schematic diagram of an image sensor and a depth sensor in a method for unlocking a door of a vehicle according to an embodiment of the present invention.
5 is a schematic diagram illustrating an example of a biometric detection method according to an embodiment of the present invention.
6 is a schematic diagram illustrating an example of determining a bio-detection result of a target object in a first image based on a first image and a second depth map in the bio-detection method according to an embodiment of the present invention.
7 is a schematic diagram of a depth estimation neural network in a method for unlocking a door of a vehicle according to an embodiment of the present invention.
8 is a schematic diagram of a relevance detection neural network in a method for unlocking a vehicle door according to an embodiment of the present invention.
9 is a schematic diagram illustrating an example of updating a depth map in a method for unlocking a door of a vehicle according to an embodiment of the present invention.
10 is a schematic diagram of peripheral pixels in a method for unlocking a door of a vehicle according to an embodiment of the present invention.
11 is another schematic diagram of surrounding pixels in a method for unlocking a door of a vehicle according to an embodiment of the present invention.
12 is another flowchart of a method for unlocking a door lock of a vehicle according to an embodiment of the present invention.
13 is a block diagram of a door unlocking device for a vehicle according to an embodiment of the present invention.
14 is a block diagram of an in-vehicle face authentication unlock system according to an embodiment of the present invention.
15 is a schematic diagram of an in-vehicle face authentication unlocking system according to an embodiment of the present invention.
16 is a schematic diagram of a vehicle according to an embodiment of the present invention.
17 is a block diagram of an electronic device 800 according to an embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Various exemplary embodiments, features and aspects of the present invention will be described in detail below with reference to the drawings. In the drawings, the same reference numerals indicate elements having the same or similar functions. Although the drawings have shown various aspects of the embodiments, the drawings are not necessarily drawn to scale unless otherwise noted.

The term "exemplary" herein means "an example, used as an embodiment, or explanatory." Here, certain embodiments described as “exemplary” should not be understood as preferred or superior to other embodiments.

The term "and/or" in the present specification simply describes a relationship with a related object, and indicates that three relationships are possible, for example, A and/or B are only A, or A and B You may show the three cases that both exist or that only B exists. In addition, the term "one or more" in the present specification indicates any one of a plurality or any combination of at least two of the plurality, for example, A, B, and C including one or more of A, B, and C It may indicate including any one or a plurality of elements selected from the set consisting of.

In addition, in order to explain the present invention more effectively, various specific details are set forth in the following specific embodiments. It should be understood by those skilled in the art that the present invention may be practiced without any specific details. In some embodiments, detailed descriptions of methods, means, elements and circuits known to those skilled in the art are not set forth in order to emphasize the spirit of the present invention.

1 is a flowchart illustrating a method for unlocking a door lock of a vehicle according to an embodiment of the present invention. This vehicle door unlocking method may be executed by a vehicle door unlocking device. For example, the door unlocking method of this vehicle can be executed by an in-vehicle device or other processing device. For example, the door unlocking device of the vehicle may be mounted on one or more of the vehicle's B-pillar, one or more doors, and one or more rearview mirrors. 2 shows a schematic diagram of a B-pillar of a vehicle. For example, the vehicle door unlocking device may be mounted 130 cm to 160 cm from the ground of the B-pillar, and the horizontal recognition distance thereof may be 30 cm to 100 cm, but the present invention is not limited thereto. 3 is a schematic diagram illustrating a mounting height and recognizable height range of a vehicle door unlocking device in a vehicle door unlocking method according to an embodiment of the present invention. In the example shown in FIG. 3 , the mounting height of the vehicle door unlocking device is 160 cm, and the recognizable height range is 140 cm to 190 cm.

In one possible embodiment, this method of unlocking a door of the vehicle may be realized by calling a computer readable instruction stored in a memory by a processor.

As shown in Fig. 1, this vehicle door unlocking method includes steps S11 to S15.

In step S11, a Bluetooth device of a predetermined identifier is searched for by a Bluetooth module mounted on the vehicle.

In one possible implementation form, searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on the vehicle is performed by a Bluetooth module mounted on the vehicle when the vehicle is in an ignition-off state or ignition-off state and is in a door-locked state. Searching for Bluetooth devices. In this embodiment, there is no need to search for the Bluetooth device of a predetermined identifier by the Bluetooth module before ignition off of the vehicle, or before ignition off of the vehicle, and also when the vehicle is in the ignition off state but the door is not locked, the Bluetooth module Since there is no need to search for a Bluetooth device with a predetermined identifier, power consumption can be further reduced.

In one possible realization, the Bluetooth module may be a Bluetooth Low Energy (BLE) module. In this realization embodiment, when the vehicle is in the ignition-off state or the ignition-off state and the door is locked, the Bluetooth module enters the broadcasting mode, so that even if it broadcasts a broadcast packet to the surroundings every predetermined period of time (for example, 100 milliseconds) do. When a surrounding Bluetooth device is executing a scanning operation, upon receiving a broadcast packet broadcast by the Bluetooth module, it transmits a scanning request to this Bluetooth module. The Bluetooth module may respond to the scan request and return a scan response packet to the Bluetooth device that sent the scan request. In this embodiment, upon receiving the scan request transmitted from the Bluetooth device of the predetermined identifier, it can be determined that the Bluetooth device of the predetermined identifier can be searched.

In another possible realization, when the vehicle is in the ignition-off state or ignition-off state, and also in the door lock state, the Bluetooth module enters a scanning state, and when scanning a Bluetooth device of a predetermined identifier, it can search for a Bluetooth device of a predetermined identifier. can decide

In one possible realization, the Bluetooth module and the face recognition module may be integrated into the face recognition system.

In another possible realization, the Bluetooth module may be separate from the face recognition system. That is, the Bluetooth module may be installed outside the face recognition system.

Although the maximum search distance of the Bluetooth module is not limited in the embodiment of the present invention, in an example, the maximum search distance may be about 30 m.

In the embodiment of the present invention, the identifier of the Bluetooth device may be a unique identifier of the Bluetooth device. For example, the identifier of the Bluetooth device may be an ID, name, or address of the Bluetooth device.

In an embodiment of the present invention, the predetermined identifier may be an identifier of a device that has succeeded in pairing with the Bluetooth module of the vehicle by the secure connection technology of Bluetooth in advance.

In the embodiment of the present invention, the number of Bluetooth devices with a predetermined identifier may be one or plural. For example, when the identifier of the Bluetooth device is the ID of the Bluetooth device, one or more Bluetooth IDs having release authority may be set in advance. For example, when the number of Bluetooth devices of a predetermined identifier is one, the Bluetooth devices of the predetermined identifier may be the Bluetooth devices of the vehicle owner. When the number of Bluetooth devices of a predetermined identifier is plural, the Bluetooth devices of the plurality of predetermined identifiers are It may include the Bluetooth device of the vehicle owner and the Bluetooth device of the vehicle owner's family, friends, and pre-registered contacts. Here, the pre-registered contact person may be a pre-registered delivery man or a staff of a real estate management company.

In the embodiment of the present invention, the Bluetooth device may be any mobile device having a Bluetooth function. For example, the Bluetooth device may be a mobile phone, a wearable device, an electronic key, or the like. Here, the wearable device may be a smart bracelet or smart glasses.

In step S12, as the Bluetooth device of the predetermined identifier can be searched, Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier is established.

In one possible implementation mode, when the number of Bluetooth devices of a predetermined identifier is plural, any one of the Bluetooth devices of a predetermined identifier can be searched for, thereby establishing Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier.

In one possible implementation mode, as the Bluetooth device of the predetermined identifier can be searched for, the Bluetooth module performs identity authentication of the Bluetooth device of the predetermined identifier, and after passing the identity authentication, the Bluetooth module and the Bluetooth device of the Bluetooth device of the predetermined identifier By establishing pairing/connection, the security of Bluetooth pairing/connection can be improved.

In step S13, the image acquisition module mounted on the vehicle is woken up and controlled so as to acquire the first image of the target object according to the success of Bluetooth pairing/connection.

In one possible implementation form, waking up and controlling the image acquisition module mounted on the vehicle to acquire the first image of the target object includes waking up the facial recognition module mounted on the vehicle, and the target by the wakeup facial recognition module and controlling the image acquisition module to acquire a first image of the object.

In the embodiment of the present invention, when a Bluetooth device of a predetermined identifier can be searched for, it indicates that a user (eg, a vehicle owner) possessing a Bluetooth device of a predetermined identifier is highly likely to enter the search range of the Bluetooth module. . At this time, as the Bluetooth device of the predetermined identifier can be searched, Bluetooth pairing/connection is established between the Bluetooth module and the Bluetooth device of the predetermined identifier, and according to the success of the Bluetooth pairing/connection, the face recognition module is woken up to the target By controlling the image acquisition module to acquire the first image of the object, the wakeup error probability of the facial recognition module can be effectively lowered by a method of waking up the facial recognition module after Bluetooth pairing/connection is successful, and the user experience can be improved. improved, and the power consumption of the face recognition module can be effectively reduced. In addition, the Bluetooth pairing/connection method has advantages in that it provides high security and supports long-distance communication compared to short-range sensor technologies such as ultrasonic waves and infrared rays. According to practice, the time it takes for a user who has a Bluetooth device with a predetermined identifier to arrive at the vehicle through this distance (the distance between the user and the vehicle when Bluetooth pairing/connection is successful), and the vehicle sleeps the face recognition module It is shown that the time to wake up to transition from the state to the active state is approximately coincident. Thereby, instead of waiting for the wakeup of the face recognition module after the user arrives at the door, when the user arrives at the door, unlocking by face recognition can be immediately performed by the wakeup face recognition module. It can improve efficiency and improve user experience. In addition, the user experience can be further improved without making the user feel Bluetooth pairing/connection. Therefore, in the embodiment of the present invention, by the method of waking up the face recognition module after successful Bluetooth pairing/connection, each aspect such as power saving, user experience and security of the face recognition module is well balanced. provides

In one possible realization, the method further includes controlling the facial recognition module to enter a sleep state when the facial image cannot be acquired within a preset time after waking up the vehicle-mounted facial recognition module. This embodiment can reduce power consumption by controlling the face recognition module to enter a sleep state when a face image cannot be acquired within a preset time after waking up the face recognition module.

In one possible realization, the method further includes controlling the facial recognition module to enter a sleep state when the face recognition module is not successfully recognized within a preset time after waking up the vehicle-mounted face recognition module. This realization can reduce power consumption by controlling the face recognition module to enter a sleep state when face recognition is not successful within a preset time after waking up the face recognition module.

In step S14, face recognition is performed based on the first image.

In one possible realization, the face recognition includes biometric detection and face authentication, and performing face recognition based on the first image is to acquire a first image by an image sensor of an image acquisition module, the first image and the pre-registered performing face authentication based on facial features, acquiring a first depth map corresponding to the first image by a depth sensor of an image acquisition module, and performing biometric detection based on the first image and the first depth map do.

In an embodiment of the present invention, the first image includes a target object. Here, the target object may be a face or at least a part of the human body, but the embodiment of the present invention is not limited thereto.

Here, the first image may be a static image or an image of a video frame. For example, an image selected from the video sequence may be a first image, and the image may be selected from the video sequence in various ways. In one embodiment, the first image is an image that satisfies a predetermined quality condition selected from a video sequence. Here, the predetermined quality conditions include whether the target object is included, whether the target object is located in the central region of the image, whether the target object is completely included in the image, the ratio of the target object to the image, and the state of the target object (eg For example, face angle), sharpness of an image, and exposure of an image may include one or any combination thereof, but embodiments of the present invention are not limited thereto.

In one example, face authentication may be performed after biometric detection. For example, when the biometric detection result of the target object indicating that the target object is a living body, the face authentication flow is triggered. On the other hand, when the biometric detection result of the target object indicating that the target object is non-living, the face authentication flow is not triggered.

In another example, biometric detection may be performed after face authentication. For example, if the face authentication succeeds, the biometric detection flow is triggered, and if the face authentication fails, the biometric detection flow is not triggered.

In another example, biometric detection and face authentication may be performed simultaneously.

In this embodiment, the biometric detection is for verifying whether the target object is a living body, for example, it can verify whether the target object is a human body. Facial authentication is for extracting facial features from an acquired image and combining the facial features from the acquired image with pre-registered facial features to determine whether they are facial features of the same person, for example, It may be determined whether a facial feature in the image is a facial feature of a vehicle owner.

In an embodiment of the present invention, the depth sensor refers to a sensor for acquiring depth information. Embodiments of the present invention do not limit the operating principle and operating wavelength range of the depth sensor.

In an embodiment of the present invention, the image sensor and the depth sensor of the image acquisition module may be separate or integrated. For example, so that the image sensor uses an RGB (Red, Red; Green, Green; Blue, Blue) sensor or an infrared sensor, while the depth sensor uses a binocular infrared sensor or a Time off Flight (TOF) sensor. They may be separate, or the image acquisition module may be integrated so as to realize the functions of the image sensor and the depth sensor by using an RGBD (Red, Red; Green, Green; Blue, Blue; Deep, depth) sensor.

As an example, the image sensor is an RGB sensor. When the image sensor is an RGB sensor, the image acquired by the image sensor is an RGB image.

As another example, the image sensor is an infrared sensor. When the image sensor is an infrared sensor, the image acquired by the image sensor is an infrared image. The infrared image may be an image with spots or an image without spots.

In another example, the image sensor may be a different type of sensor, but the embodiment of the present invention is not limited thereto.

Optionally, the door unlocking device of the vehicle may acquire the first image in various ways. For example, in some embodiments, a camera is installed in a door unlocking device of a vehicle, and the vehicle door unlocking device acquires a static image or a video stream by means of the camera to obtain a first image. Embodiments of the present invention do not limit this.

As an example, the depth sensor is a three-dimensional sensor. For example, the depth sensor is a binocular infrared sensor comprising two infrared cameras, a time-of-flight TOF sensor or a structured light sensor. The structured light sensor may be a coded structured light sensor or a speckle structured light sensor. By acquiring the depth map of the target object with the depth sensor, it is possible to acquire a high-precision depth map. According to the embodiment of the present invention, by performing biometric detection using a depth map including the target object, depth information of the target object can be sufficiently obtained and the accuracy of biometric detection can be increased. In the embodiment of the present invention, for example, when the target object is a face, by performing biometric detection using a depth map including the face, depth information of face data can be sufficiently found, and the accuracy of biometric face detection can be improved. can

In one example, the TOF sensor uses a TOF module based on the infrared wavelength range. In this example, by using the TOF module based on the infrared wavelength range, it is possible to reduce the influence of external light on the imaging of the depth map.

In the embodiment of the present invention, the first depth map and the first image correspond to each other. For example, the first depth map and the first image are acquired for the same scene by the depth sensor and the image sensor, respectively, or are acquired for the same target area at the same time by the depth sensor and the image sensor, Embodiments of the present invention do not limit this.

4A is a schematic diagram of an image sensor and a depth sensor in a method for unlocking a door of a vehicle according to an embodiment of the present invention. In the example shown in FIG. 4A, the image sensor is an RGB sensor, the camera of the image sensor is an RGB camera, the depth sensor is a binocular infrared sensor, and two infrared (IR) cameras installed on both sides of the RGB camera of the image sensor are included. . Two infrared cameras acquire depth information based on the principle of binocular disparity.

In one example, the image acquisition module is installed between the infrared camera of the binocular infrared sensor and the camera of the image sensor, and further includes one or more beaming lights including at least one of a beaming light for an image sensor and a beaming light for a depth sensor. For example, when the image sensor is an RGB sensor, the beam light for the image sensor is a white light, when the image sensor is an infrared sensor, the beam light for the image sensor is an infrared light, and when the depth sensor is a binocular infrared sensor, the depth of field The light beam for sensors may be an infrared light. In the example shown in FIG. 4A, an infrared light is provided between the infrared camera of a binocular infrared sensor and the camera of an image sensor. For example, you may make it use infrared rays of 940 nm as an infrared light.

In one example, the light beam light may be in the normally open mode. In this example, when the camera of the image acquisition module is in the operating state, the beam light is in the on state.

In another example, when the light beam is insufficient, the light beam light may be turned on. For example, the ambient light intensity may be acquired by the ambient light sensor, and when the ambient light intensity is lower than the light intensity threshold, it may be determined that the light beam is insufficient, and the auxiliary light may be turned on.

4B is another schematic diagram of an image sensor and a depth sensor in a method for unlocking a door of a vehicle according to an embodiment of the present invention. In the example shown in FIG. 4B , the image sensor is an RGB sensor, the camera of the image sensor is an RGB camera, and the depth sensor is a TOF sensor.

In one example, the image acquisition module further includes a laser installed between the camera of the depth sensor and the camera of the image sensor. For example, a laser is installed between the camera of the TOF sensor and the camera of the RGB sensor. For example, the laser may be a VCSEL (Vertical Cavity Surface Emitting Laser), and the TOF sensor may acquire a depth map by laser light emitted from the VCSEL.

In an embodiment of the present invention, the depth sensor is for acquiring a depth map, and the image sensor is for acquiring a two-dimensional image. Although the image sensor is described by taking the RGB sensor and the infrared sensor as examples, and the depth sensor is described by using the binocular infrared sensor, the TOF sensor, and the structured light sensor as examples, it can be understood by those skilled in the art that the embodiment of the present invention is not limited thereto. Those skilled in the art can select the types of the image sensor and depth sensor according to the actual application requirements, and it is sufficient if it can realize the acquisition of a two-dimensional image and a depth map, respectively.

In step S15, a door unlock command and/or a door open command are transmitted to one or more doors of the vehicle according to the success of face recognition.

The door of the embodiment of the present invention may include a door for entry and exit (eg, a left front door, a right front door, a left rear door, a right rear door), or may include a trunk door of a vehicle. Accordingly, the one or more door locks may include one or more of a left front door lock, a right front door lock, a left rear door lock, a right rear door lock, and a trunk door lock.

In one possible realization, sending a door unlock command and/or a door open command to one or more doors of the vehicle according to the success of facial recognition comprises: obtaining status information of one or more doors of the vehicle according to the success of facial recognition; , when the status information of the door is unlocked, sending a door unlocking command and a door open command to the door; and sending a door open command to the door.

In one possible implementation, sending a door unlock command and/or a door open command to one or more doors of the vehicle upon success of face recognition comprises determining which door is authorized to unlock on a target upon success of face recognition; , sending a door unlocking command and/or a door opening command to one or more doors of the vehicle according to the door to which the target object is granted unlocking authority. For example, all doors may be sufficient as the door to which unlocking authority is given to a target object, and a trunk door may be sufficient as it.

For example, the door to which the vehicle owner or the vehicle owner's family and friends have unlocking authority may be all doors, and the door to which the delivery person or the staff of the real estate management company has unlocking authority may be a trunk door. Here, the vehicle owner may set the door information having unlocking authority to another person. In addition, for example, in the scene of a vehicle dispatch service, the doors to which the passenger has unlocking authority may be doors other than the driver's seat and trunk doors. When the door to which the target object is granted the unlocking authority is a trunk door, a door unlocking command may be transmitted to the trunk door lock.

In one example, when the door to which the unlocking authority is granted to the target object is included in only the trunk door, the door closing command may be transmitted after a predetermined time after transmitting the door unlocking command to the trunk door lock. For example, the predetermined time may be 3 minutes. For example, if the door for which the delivery person has the right to unlock includes only the trunk door, the delivery person's request to put the luggage in the trunk by sending the door closing command 3 minutes after sending the door unlock command to the trunk door lock It can not only respond to this, but also increase the security of the vehicle.

In one possible implementation mode, in addition to determining the door to which the unlocking authority is granted to the target object, the time period for which the unlocking authority is granted to the target object, the number of times of the unlocking authority corresponding to the target object, etc. may be determined.

For example, the time period for which the unlocking authority is granted to the target object may be any period or a predetermined period. For example, the time period during which the vehicle owner or the vehicle owner's family has the right to release may be any period. The vehicle owner may set a time for which another person has the right to unlock. For example, in a scene in which the vehicle owner lends the vehicle to a friend, the time during which the friend has the unlocking authority may be set to 2 days. Also, for example, the vehicle owner may set the time for the delivery person to have the unlocking authority from 13:00 to 14:00 on September 29, 2019 after receiving a call from the delivery person. In addition, for example, in the case of a rental car scene, when a customer rents a vehicle for 3 days, the rental car staff may set the time for which the customer has the unlocking authority to 3 days. In addition, for example, in the scene of the dispatch service, the time for which the passenger has the unlocking authority may be the service period described in the travel order form.

For example, the number of times of the unlock authority corresponding to the target object may be an infinite number of times or a finite number of times. For example, the number of corresponding unlocking rights of the vehicle owner or the vehicle owner's family and friends may be infinite. Further, for example, the number of times of the corresponding unlocking authority of the delivery person may be a finite number of times, for example, once.

For example, the SoC of the vehicle's door unlocking device may control the door to be unlocked by sending a door unlocking command to the door domain controller.

In one possible realization, performing the biometric detection based on the first image and the first depth map includes updating the first depth map based on the first image to obtain a second depth map, and the first image and the second depth map and determining a biometric detection result of the target object based on the .

Specifically, the second depth map is obtained by updating the depth values of one or a plurality of pixels in the first depth map based on the first image.

In some embodiments, a second depth map is obtained by updating a depth value of an effective depth pixel in the first depth map based on the first image.

Here, the effective depth pixel in the depth map may refer to a pixel included in the depth map and having an invalid depth value, that is, a pixel having an inaccurate depth value or a depth value that is not clear in reality. The number of depth invalid pixels may be one or plural. By updating the depth values of one or more effective depth pixels in the depth map, the depth values of the effective depth pixels become more accurate, contributing to the improvement of the accuracy of biometric detection.

In some embodiments, the first depth map is a depth map with missing values, and a second depth map is obtained by repairing the first depth map based on the first image. Here, optionally, the restoration of the first depth map includes determining or supplementing the depth value of the missing pixel, but the embodiment of the present invention is not limited thereto.

In an embodiment of the present invention, the first depth map may be updated or restored in various ways. In some embodiments, biometric detection is performed using the first image directly, for example, the first depth map is updated using the first image directly. In some other embodiments, the first image is pre-processed, and biometric detection is performed based on the first image after the pre-processing. For example, an image of the target object is acquired from the first image, and the first depth map is updated based on the image of the target object.

The image of the target object may be cut out from the first image in various ways. As an example, target detection is performed on the first image to obtain position information of the target object, for example, position information of a bounding box of the target object, and based on the position information of the target object, the target in the first image Crop the image of the object. For example, an image of a region in which a bounding box of the target object is located in the first image is cut out as an image of the target object. Further, for example, the bounding box of the target object is enlarged by a constant multiple, and an image of a region in which the enlarged bounding box is located is cut out as an image of the target object from the first image. As another example, information of a key point of the target object in the first image is acquired, and an image of the target object is acquired in the first image based on the information of the key point of the target object.

Optionally, target detection is performed on the first image to obtain position information of an area in which the target object is located, and keypoint detection is performed on the image of the area in which the target object is located so that the keypoint of the target object in the first image is obtained. get information

Optionally, the information of the key points of the target object may include position information of a plurality of key points of the target object. When the target object is a face, the key points of the target object may include one or more of a key point of an eye, a key point of an eyebrow, a key point of a nose, a key point of a mouth, a key point of a face outline, and the like. Here, the key point of the eye may include one or more of a key point of an eye outline, a key point of an eye tail, and a key point of a pupil.

In one example, an outline of the target object is determined based on information of key points of the target object, and an image of the target object is cut out from the first image based on the outline of the target object. Compared with the positional information of the target obtained by target detection, the position of the target obtained by the keypoint information is more accurate, contributing to the improvement of the accuracy of subsequent biometric detection.

Optionally, determine a contour of the target object in the first image based on a keypoint of the target object in the first image, and an image of a region in which the contour of the target object in the first image is located or the contour of the target object An image of a region in which the region to be located is enlarged by a predetermined multiple may be determined as the image of the target object. For example, an elliptical area determined based on the keypoint of the target object in the first image is determined as the image of the target object, or the smallest circumscribed rectangular area of the elliptical area determined based on the keypoint of the target object in the first image is the target object may be determined as an image of , but the embodiment of the present invention is not limited thereto.

In this way, by acquiring the image of the target object from the first image and performing biometric detection based on the image of the target object, it is possible to reduce interference with the biometric detection by the background information in the first image.

In the embodiment of the present invention, update processing may be performed on the acquired original depth map. Alternatively, in some embodiments, a depth map of the target object is obtained from the first depth map, and the depth map of the target object is updated based on the first image to obtain a second depth map.

As an example, position information of the target object in the first image is acquired, and a depth map of the target object is acquired from the first depth map based on the position information of the target object. Optionally, registration or positioning processing may be performed in advance for the first depth map and the first image. Embodiments of the present invention do not limit this.

In this way, by acquiring the depth map of the target object from the first depth map, updating the depth map of the target object based on the first image to obtain the second depth map, the living body based on the background information in the first depth map Interference with detection can be reduced.

In some embodiments, after the first image and the first depth map corresponding to the first image are acquired, the first image and the first depth map are aligned based on the parameters of the image sensor and the depth sensor parameters.

As an example, by performing the conversion process on the first depth map, the first depth map after the conversion process and the first image may be aligned. For example, the first transformation matrix may be determined based on the parameters of the depth sensor and the parameters of the image sensor, and the transformation processing may be performed on the first depth map based on the first transformation matrix. Accordingly, based on at least a part of the first image, at least a part of the first depth map after the conversion process may be updated to obtain the second depth map. For example, based on the first image, the first depth map after the conversion process may be updated to obtain the second depth map. Also, for example, based on the image of the target object cut out from the first image, the depth map of the target object cut out from the first depth map may be updated to obtain a second depth map.

As another example, by performing the conversion process on the first image, the first image after the conversion process and the first depth map may be aligned. For example, the second transformation matrix may be determined based on the parameters of the depth sensor and the parameters of the image sensor, and the transformation processing may be performed on the first image based on the second transformation matrix. Accordingly, based on at least a part of the first image after the conversion process, at least a part of the first depth map may be updated to obtain the second depth map.

Optionally, the parameters of the depth sensor may include internal parameters and/or external parameters of the depth sensor, and the parameters of the image sensor may include internal parameters and/or external parameters of the image sensor. By aligning the positions of the first depth map and the first image, the positions of the corresponding portions of the first depth map and the first image in the first depth map and the first image may be the same.

In the above examples, the first image is an original image (eg, an RGB or infrared image), but in some other embodiments, the first image may be an image of a target object cut out from the original image. Similarly, the first depth map may be a depth map of the target object cut out from the original depth map. Examples of the present invention do not limit these.

5 is a schematic diagram illustrating an example of a biometric detection method according to an embodiment of the present invention. In the example shown in Fig. 5, the first image is an RGB image, and the target object is a face, and alignment/correction processing is performed on the RGB image and the first depth map, and the processed image is input to the face keypoint model and processed. , obtain an RGB face image (image of the target object) and a depth face image (depth map of the target object), and update or repair the depth face image based on the RGB face image. Thereby, the amount of subsequent data processing can be reduced, and biometric detection efficiency and accuracy can be increased.

In an embodiment of the present invention, the biodetection result of the target object may indicate that the target object is a living body or that the target object is a non-living object.

In some embodiments, the first image and the second depth map are input to the biodetection neural network and processed, and a biometric detection result of the target object in the first image is obtained. Alternatively, the biometric detection result is obtained by processing the first image and the second depth map by another biometric detection algorithm.

In some embodiments, feature extraction processing is performed on the first image to obtain first feature information, and feature extraction processing is performed on the second depth map to obtain second feature information, based on the first feature information and the second feature information to determine the biodetection result of the target object in the first image.

Here, the feature extraction process selectively can be realized by a neural network or other machine learning algorithm, and the type of the extracted feature information can be selectively obtained by learning using a sample, but the embodiment of the present invention is do not limit them

In some specific scenes (eg, a scene with strong outdoor light), a partial area of an acquired depth map (eg, a depth map acquired by a depth sensor) may become effective. In addition, even in normal light irradiation, partial ineffectiveness of the depth map is randomly caused by factors such as reflection by glasses, dark hair, or black glasses frame. Some special papers also cause large area or partial ineffectiveness in depth maps of similarly printed face photos. In addition, when the active light source of the depth sensor is blocked, a part of the depth map is invalid, but the image formation in the non-living image sensor is normal. In this way, when a part or all of the depth map becomes invalid, an error occurs in the identification of a living body and a non-living body based on the depth map. Accordingly, in the embodiment of the present invention, by repairing or updating the first depth map and performing biometric detection using the repaired or updated depth map, it contributes to the improvement of accuracy of biometric detection.

6 is a schematic diagram illustrating an example of determining a biometric detection result of a target object in a first image based on a first image and a second depth map by the biometric detection method according to an embodiment of the present invention.

In this example, the first image and the second depth map are input to the biometric detection network, and biometric detection processing is performed to obtain a biometric detection result.

As shown in Fig. 6, this biometric detection network performs feature extraction processing on a first image to obtain first characteristic information, and a first subnetwork performs feature extraction processing on a second depth map to obtain second characteristic information. It includes two branches of the second subnetwork to obtain

In an optional example, the first subnetwork may include a convolutional layer, a downsampling layer and a full coupling layer.

For example, the first subnetwork may include a convolutional layer of one layer, a downsampling layer of one layer, and an entire coupling layer of the first layer. Here, the convolutional layer of this first layer may include one or a plurality of convolutional layers, the downsampling layer of this first layer may include one or a plurality of downsampling layers, and the entire coupling layer of the first layer may include one or a plurality of downsampling layers. A plurality of total bonding layers may be included.

Further, for example, the first subnetwork may include a multi-layered convolutional layer, a multi-layered downsampling layer, and a single-layered total coupling layer. Here, the convolutional layer of each layer may include one or a plurality of convolutional layers, and the downsampling layer of each layer may include one or a plurality of downsampling layers, and the total coupling layer of this first layer may include one or a plurality of downsampling layers. The entire bonding layer may be included. where the downsampling layer of the ith layer is cascaded after the convolutional layer of the ith layer, the convolutional layer of the i+1th layer is cascaded after the downsampling layer of the ith layer, and the total coupling after the downsampling layer of the nth layer is The layers are cascaded. Here, i and n are both positive integers, 1≤i≤n, and n is the number of convolutional and downsampling layers in the depth estimation neural network.

Alternatively, the first subnetwork may include a convolutional layer, a downsampling layer, a normalization layer, and an overall combining layer.

For example, the first subnetwork may include one convolutional layer, one normalization layer, one downsampling layer, and one full coupling layer. F, the convolutional layer of this first layer may include one or a plurality of convolutional layers, the downsampling layer of this first layer may include one or a plurality of downsampling layers, and the total coupling layer of this first layer is one or A plurality of total bonding layers may be included.

Also, for example, the first subnetwork may include a multi-layered convolutional layer, a plurality of normalization layers, a multi-layered downsampling layer, and one entire coupling layer. Here, the convolutional layer of each layer may include one or a plurality of convolutional layers, and the downsampling layer of each layer may include one or a plurality of downsampling layers, and the total coupling layer of this first layer may include one or a plurality of downsampling layers. The entire bonding layer may be included. Here, the i-th normalization layer is cascaded after the convolutional layer of the i-th layer, the down-sampling layer of the i-th layer is cascaded after the i-th normalization layer, and the convolutional layer of the i+1th layer is after the down-sampling layer of the i-th layer. are cascaded, and the entire coupling layer is cascaded after the downsampling layer of the nth layer. Here, i and n are both positive integers, 1≤i≤n, and n represents the number of convolutional layers, downsampling layers and the number of normalization layers in the first subnetwork.

As an example, convolution processing is performed on the first image to obtain a first convolution result, down-sampling processing is performed on the first convolution result to obtain a first down-sampling result, and a first down-sampling result is obtained based on the first down-sampling result. 1 Acquire feature information.

For example, the convolution process and the downsampling process may be performed on the first image by the first convolutional layer and the first downsampling layer. The one-layer convolutional layer may include one or a plurality of convolutional layers, and the one-layer downsampling layer may include one or a plurality of downsampling layers.

Further, for example, the convolutional processing and the downsampling processing may be performed on the first image by the multilayered convolutional layer and the multilayered downsampling layer. Here, the convolutional layer of each layer may include one or a plurality of convolutional layers, and the downsampling layer of each layer may include one or a plurality of downsampling layers.

For example, performing down-sampling processing on the first convolution result to obtain a first down-sampling result includes performing normalization processing on the first convolution result to obtain a first normalization result, and down-sampling the first normalization result. You may make it include performing a sampling process and obtaining a 1st down-sampling result.

For example, the first downsampling result may be input to all the coupling layers, and the first downsampling result may be fused by all the coupling layers to obtain the first characteristic information.

Optionally, the second subnetwork has the same network structure as the first subnetwork but has different parameters. Alternatively, the second sub-network has a network structure different from that of the first sub-network. Embodiments of the present invention do not limit this.

As shown in FIG. 6 , the biometric detection network processes the first characteristic information obtained by the first subnetwork and the second characteristic information obtained by the second subnetwork to obtain the biometric detection result of the target object in the first image. It further includes a third subnetwork for obtaining. Optionally, the third subnetwork may include an entire coupling layer and an output layer. For example, the output layer employs a softmax function, and when the output of the output layer is 1, it indicates that the target object is a living body, and when the output of the output layer is 0, it indicates that the target object is non-living. There is no limitation on the specific realization of the third subnetwork.

As an example, fusion processing is performed on the first characteristic information and the second characteristic information to obtain third characteristic information, and a biometric detection result of the target object in the first image is determined based on the third characteristic information.

For example, fusion processing is performed on the first characteristic information and the second characteristic information by the entire coupling layer to obtain the third characteristic information.

In some embodiments, a probability that the target object in the first image is a living body is obtained based on the third characteristic information, and a biometric detection result of the target object is determined based on the probability that the target object is a living body.

For example, when the probability that the target object is a living body is greater than the second threshold value, it is determined that the target object is a living body as a result of detecting the target object as a living body. Further, for example, when the probability that the target object is a living body is equal to or less than the second threshold value, it is determined that the target object is non-living as a result of detecting the biological body.

In some other exemplary embodiments, a probability that the target object is non-living is obtained based on the third characteristic information, and a biometric detection result of the target object is determined based on the probability that the target object is non-living. For example, when the probability that the target object is nonliving is greater than the third threshold value, it is determined that the target object is nonliving as a result of biodetection of the target object. Also, for example, when the probability that the target object is non-living is equal to or less than the third threshold value, it is determined as a living body detection result of the target object.

In one example, the third characteristic information may be input to the Softmax layer so that the Softmax layer obtains the probability that the target object is living or non-living. For example, the output of the Softmax layer includes two types of neurons: a neuron indicating a probability that the target object is a living body and a neuron indicating a probability that the target object is non-living, but the embodiment of the present invention is not limited thereto.

In an embodiment of the present invention, a first image and a first depth map corresponding to the first image are acquired, and the first depth map is updated based on the first image to obtain a second depth map, and the first image and the second By determining the bio-detection result of the target object in the first image based on the depth map, the depth map may be improved and accuracy of bio-detection may be increased.

In one possible embodiment, updating the first depth map based on the first image to obtain the second depth map comprises depth estimates of a plurality of pixels in the first image based on the first image and the association of the plurality of pixels with each other. determining related information representing the figure, and updating the first depth map to obtain a second depth map based on the estimated depth values of the plurality of pixels and the related information.

Specifically, depth estimation values of a plurality of pixels in the first image are determined based on the first image, and the first depth map is repaired and improved based on the depth estimation values of the plurality of pixels.

Specifically, the first image is processed to obtain depth estimates of a plurality of pixels in the first image. For example, a first image is input to a depth estimation neural network and processed, and a depth estimation result of a plurality of pixels is obtained, for example, an estimated depth map corresponding to the first image is obtained; does not limit this.

In some embodiments, depth estimates of a plurality of pixels in the first image are determined based on the first image and the first depth map.

As an example, a first image and a first depth map are input to a depth estimation neural network for processing, and depth estimates of a plurality of pixels in the first image are obtained. Alternatively, depth estimation values of a plurality of pixels are obtained by processing the first image and the first depth map by another method, but the embodiment of the present invention is not limited thereto.

7 is a schematic diagram of a depth estimation neural network in a method for unlocking a door of a vehicle according to an embodiment of the present invention. As shown in FIG. 7 , the first image and the first depth map may be input to a depth estimation neural network and processed to obtain an initial depth estimation map. Based on the initial depth estimation map, depth estimation values of the plurality of pixels in the first image may be determined. For example, the pixel value of the initial depth estimation map is the depth estimate of the corresponding pixel in the first image.

The depth estimation neural network can be realized by various network structures. In one example, the depth estimation neural network includes a coded part and a decode part. Here, optionally, the coding part may include a convolutional layer and a downsampling layer, and the decoding part may include a deconvolutional layer and/or an upsampling layer. In addition, the encoding part and/or the decoding part may further include a normalization layer. The embodiment of the present invention does not limit the specific realization of the coded part and the decode part. In the coding part, as the number of network layers increases, the resolution of the feature map gradually decreases, and the quantity of the feature map gradually increases, thereby obtaining rich semantic features and spatial features of the image. In the decode part, the resolution of the feature map is gradually increased, and the resolution of the feature map finally output in the decode part becomes the same as the resolution of the first depth map.

In some embodiments, fusion processing is performed on the first image and the first depth map to obtain a fusion result, and depth estimates of a plurality of pixels in the first image are determined based on the fusion result.

In one example, the first image and the first depth map may be concatted to obtain a fusion result.

In one example, convolution processing is performed on the fusion result to obtain a second convolution result, down-sampling processing is performed based on the second convolution result to obtain a first coded result, and a first image is obtained based on the first coded result. Determining depth estimates of a plurality of pixels in .

For example, the convolution layer may perform convolution processing on the fusion result to obtain a second convolution result.

For example, normalization processing is performed on the second convolution result to obtain a second normalization result, and downsampling processing is performed on the second normalization result to obtain a first coded result. Here, the normalization layer may perform normalization processing on the second convolution result to obtain the second normalization result, and the downsampling layer may perform downsampling processing on the second normalization result to obtain the first coded result. Alternatively, the down-sampling layer may perform down-sampling processing on the second convolution result to obtain the first coded result.

For example, deconvolution processing is performed on the first encoding result to obtain a first deconvolution result, and normalization processing is performed on the first deconvolution result to obtain a depth estimation value. Here, the deconvolution layer performs deconvolution processing on the first coded result to obtain the first deconvolution result, and the normalization layer performs normalization processing on the first deconvolution result to obtain a depth estimation value. do. Alternatively, the depth estimation value may be obtained by performing deconvolution processing on the first coded result by the deconvolution layer.

For example, an upsampling process is performed on the first coding result to obtain a first upsampling result, and a normalization process is performed on the first upsampling result to obtain a depth estimation value. Here, the up-sampling layer may perform up-sampling processing on the first coding result to obtain the first up-sampling result, and the normalization layer may perform normalization processing on the first up-sampling result to obtain a depth estimation value. Alternatively, the upsampling layer may perform an upsampling process on the first coding result to obtain a depth estimation value.

Further, the first image is processed to obtain related information of a plurality of pixels in the first image. Here, the related information of the plurality of pixels in the first image may include a degree of relation between each of the plurality of pixels of the first image and its surrounding pixels. Here, the neighboring pixels of a pixel may include one or more adjacent pixels adjacent to the pixel, or may include a plurality of pixels whose distance from the pixel does not exceed a certain numerical value. For example, as shown in FIG. 10 , the peripheral pixels of the pixel 5 are adjacent thereto: the pixel 1 , the pixel 2 , the pixel 3 , the pixel 4 , the pixel 6 , and the pixel 7 . ), a pixel (8) and a pixel (9), so that the relevant information of a plurality of pixels in the first image is: a pixel (1), a pixel (2), a pixel (3), a pixel (4); The relationship between the pixel 6 , the pixel 7 , the pixel 8 , and the pixel 9 and the pixel 5 is included. As an example, the degree of correlation between the first pixel and the second pixel may be measured by the correlation between the first pixel and the second pixel. An embodiment of the present invention may determine the correlation between pixels by using a related technique, and a detailed description thereof will be omitted.

In an embodiment of the present invention, related information of a plurality of pixels may be determined by various methods. In some embodiments, the first image may be input to the relevance detection neural network to be processed, and related information of a plurality of pixels in the first image may be obtained. For example, a related feature map corresponding to the first image is obtained. Alternatively, the related information of a plurality of pixels may be obtained by another algorithm. Embodiments of the present invention do not limit this.

8 is a schematic diagram of a relevance detection neural network in a method for unlocking a vehicle door according to an embodiment of the present invention. As shown in Fig. 8, the first image is input to the relevance detection neural network and processed to obtain a plurality of related feature maps. Based on the plurality of related feature maps, related information of a plurality of pixels in the first image may be determined. For example, a peripheral pixel of a certain pixel indicates a pixel whose distance to this pixel is 0, that is, a peripheral pixel of this pixel indicates a pixel adjacent to this pixel. In this case, the relevance detection neural network may output 8 related feature maps. For example, in the first related feature map, the pixel value of the pixel (P _i , _j ) = the degree of relevance between the pixel (P _i-1 , _j-1 ) and the pixel (P _i , _j ) of the first image, where (P _i , _j ) represents the pixel in the i-th row and the j-th column, and in the second related feature map, the pixel value of the pixel (P _i , _j ) = the pixel (P _i-1 , _j ) of the first image and It is the degree of relevance of the pixel (P _i , _j ), and in the third related feature map, the pixel value of the pixel (P _i , _j ) = the pixel (P _i-1 , _j+1 ) and the pixel (P _i , _j ) of the first image ), and in the fourth related feature map, the pixel value of the pixel (P _i , _j ) = the relation between the pixel (P _i , _j-1 ) and the pixel (P _i , _j ) of the first image, In the fifth related feature map, the pixel value of the pixel (P _i , _j ) = the pixel value of the first image (P _i , _j+1 ) and the pixel (P _i , _j ), and in the sixth related feature map, the pixel value The pixel value of (P _i , _j ) = the pixel of the first image (P _i+1 , _j-1 ) and the pixel (P _i , _j ) are related, and in the related feature map, the pixel of the pixel (P _i , _j ) Value = the relation between the pixel (P _i+1 , _j ) of the first image and the pixel (P _i , _j ), and in the eighth related feature map, the pixel value of the pixel (P _i , _j ) = the pixel of the first image ( P _i+1 , _j+1 ) and the pixel (P _i , _j ) are related.

The relevance detection neural network can be realized by various network structures. As an example, the relevance detection neural network may include a coded portion and a decoded portion. Here, the coding part may include a convolutional layer and a downsampling layer, and the decoding part may include a deconvolutional layer and/or an upsampling layer. The encoding portion may further include a normalization layer, and the decoding portion may include a normalization layer. In the coding part, the resolution of the feature map gradually decreases, and the number of feature maps gradually increases, thereby obtaining rich semantic features and spatial features of the image. The resolution of the feature map gradually increases in the decode part, and in the decode part. The resolution of the finally output feature map becomes the same as the resolution of the first image. In an embodiment of the present invention, the related information may be an image or other data format, for example, a matrix or the like.

As an example, inputting and processing a first image into a relevance detection neural network, and obtaining relevant information of a plurality of pixels in the first image, is equivalent to performing convolution processing on the first image to obtain a third convolution result. , performing downsampling processing based on the third convolution result to obtain a second coding result, and obtaining related information of a plurality of pixels in the first image based on the second coding result.

In one example, the convolution layer may perform convolution processing on the first image to obtain a third convolution result.

In one example, performing down-sampling processing based on the third convolution result to obtain a second coded result is performing normalization processing on the third convolution result to obtain a third normalization result, and obtaining a third normalization result is down-sampling. The sampling process may be performed to obtain a second encoding result. In this example, the normalization layer performs normalization processing on the third convolution result to obtain the third normalization result, and the downsampling layer performs downsampling processing on the third normalization result to obtain the second coding result. do. Alternatively, the down-sampling layer may perform down-sampling processing on the third convolution result to obtain the second coded result.

In one example, determining the relevant information based on the second encoding result includes performing deconvolution processing on the second encoding result to obtain a second deconvolution result, and performing normalization processing on the second deconvolution result to be related It may include obtaining information. In this example, the deconvolution layer performs deconvolution processing on the second coded result to obtain a second deconvolution result, and the normalization layer performs normalization processing on the second deconvolution result to obtain related information. you can make it Alternatively, the deconvolution layer may perform deconvolution processing on the second coded result to obtain related information.

In one example, determining the related information based on the second encoding result is performing upsampling processing on the second encoding result to obtain a second upsampling result, and performing normalization processing on the second upsampling result to obtain related information may be included. In this example, the up-sampling layer may perform up-sampling processing on the second coding result to obtain the second up-sampling result, and the normalization layer may perform normalization processing on the second up-sampling result to obtain related information. Alternatively, the upsampling layer may perform upsampling processing on the second coding result to obtain related information.

Current 3D sensors such as TOF and structured light are easily affected by sunlight when outdoors, so large areas of holes/deletions occur in the depth map, affecting the performance of 3D biometric detection algorithms. In the embodiment of the present invention, the 3D biometric detection algorithm based on the self-improvement of the depth map improves the depth map detected by the 3D sensor and repairs it, thereby increasing the performance of the 3D biometric detection algorithm.

In some embodiments, after depth estimation values and related information of the plurality of pixels are obtained, the first depth map is updated based on the depth estimation values and related information of the plurality of pixels to obtain a second depth map. 9 is a schematic diagram illustrating an example of updating a depth map in a method for unlocking a door of a vehicle according to an embodiment of the present invention. In the example shown in FIG. 9 , the first depth map is a depth map with missing values, and the obtained depth estimation values and related information of a plurality of pixels are initial depth estimation maps and related feature maps, respectively. In this case, the depth map with the missing value, the initial depth estimation map, and the related feature map are input to the depth map update module (for example, the depth update neural network) and processed, and the final depth map, that is, the second depth map get

In some embodiments, the depth estimation value of the depth effective pixel and the depth estimation value of a plurality of neighboring pixels of the depth effective pixel are obtained from the depth estimation value of the plurality of pixels, and the depth effective pixel and the depth effective pixel are obtained from the related information of the plurality of pixels. Acquire the degree of relevance of a plurality of peripheral pixels, and based on the depth estimate value of the depth effective pixel, the depth estimate value of the plurality of peripheral pixels of the depth effective pixel, and the degree of relevance between the depth effective pixel and the peripheral pixel of the depth effective pixel, the depth effective pixel Determines the depth value after updating the pixel.

In an embodiment of the present invention, the effective depth pixel may be determined from the depth map by various methods. As an example, a pixel having a depth value of 0 in the first depth map is set as an effective depth pixel, or a pixel having no depth value in the first depth map is set as an effective depth pixel.

In this example, for a portion with a value (that is, a depth value not equal to 0) in the first depth map with a missing value, the depth value is considered accurate and reliable, and this portion is not updated and the original Although the depth value of is maintained, the depth value is updated for a pixel whose depth value becomes 0 in the first depth map.

As another example, the depth sensor may set one or more predetermined numerical values, numerical values, or predetermined ranges for the depth values of the effective depth pixels. In this example, in the first depth map, a pixel having a depth value equal to or within a predetermined range may be determined as an effective depth pixel.

In an embodiment of the present invention, the effective depth pixel in the first depth map may be determined by another statistical method, but the present invention is not limited thereto.

In this realization embodiment, the depth value of the pixel at the same position as the depth effective pixel in the first image may be determined as the depth estimate value of the depth effective pixel, and similarly, pixels at the same position as the peripheral pixels of the depth effective pixel in the first image The depth value of may be determined as the depth estimation value of pixels surrounding the depth effective pixel.

As an example, the distance between the peripheral pixel of the depth effective pixel and the depth effective pixel is less than or equal to the first threshold.

10 is a schematic diagram of peripheral pixels in a method for unlocking a door of a vehicle according to an embodiment of the present invention. For example, when the first threshold value is 0, only adjacent pixels are set as peripheral pixels. For example, the adjacent pixels of pixel 5 are pixel 1 , pixel 2 , pixel 3 , pixel 4 , pixel 6 , pixel 7 , pixel 8 , and pixel 9 . ), pixel (1), pixel (2), pixel (3), pixel (4), pixel (6), pixel (7), pixel (8), and pixel (9) of pixel (5)　 with surrounding pixels.

11 is another schematic diagram of peripheral pixels in a method for unlocking a door of a vehicle according to an embodiment of the present invention. For example, when the first threshold value is 1, in addition to the adjacent pixel, pixels adjacent to the adjacent pixel are also regarded as adjacent pixels. That is, in addition to the pixel (1), the pixel (2), the pixel (3), the pixel (4), the pixel (6), the pixel (7), the pixel (8), and the pixel (9), the pixel 10 - the pixel ( 25) is also referred to as a peripheral pixel of the pixel 5 .

As an example, based on the depth estimation value of the peripheral pixel of the depth effective pixel and the degree of relationship between the depth effective pixel and the plurality of peripheral pixels of the depth effective pixel, a depth-related value of the depth effective pixel is determined, and the depth estimated value of the depth effective pixel and Based on the depth-related value, a depth value after updating of the effective depth pixel is determined.

As another example, based on the depth estimation value of the peripheral pixel of the depth effective pixel and the relationship between the depth effective pixel and the peripheral pixel, an effective depth value of the peripheral pixel for the depth effective pixel is determined, and the effective depth value for the depth effective pixel is determined. Based on the effective depth value of each neighboring pixel and the depth estimation value of the depth effective pixel, the updated depth value of the depth effective pixel is determined. For example, the product of the estimated depth value of one neighboring pixel of the depth effective pixel and the degree of relevance corresponding to the neighboring pixel may be determined as the effective depth value of this neighboring pixel with respect to the depth effective pixel. Here, the degree of relevance corresponding to the surrounding pixel refers to the degree of relevance between the surrounding pixel and the effective depth pixel. For example, the first product is obtained by determining the product of the first predetermined coefficient and the sum of the effective depth values of the respective neighboring pixels with respect to the depth effective pixel, and the product of the depth estimated value of the depth effective pixel and the second predetermined coefficient is determined to obtain a second product The product may be obtained, and the sum of the first product and the second product may be determined as the depth value after the depth effective pixel is updated. In some embodiments, the sum of the first predetermined coefficient and the second predetermined coefficient is 1.

In one example, a weighted sum of depth estimated values of a plurality of peripheral pixels of the depth effective pixel is calculated using the degree of relevance between the depth effective pixel and each peripheral pixel as the weight of each peripheral pixel to obtain a depth related value of the depth effective pixel. For example, when the pixel 5 is a depth effective pixel, the depth-related value of the depth effective pixel 5 is

, and the depth value F ₅ ′ after the depth effective pixel 5 is updated can be determined by Equation 1 .

only,

, w _i represents the degree of relevance between the pixel (i) and the pixel (5), and F _i represents the estimated depth of the pixel (i).

In another example, the product of each of the plurality of peripheral pixels of the depth effective pixel, the degree of relevance of the depth effective pixel, and the estimated depth of each peripheral pixel is determined, and the maximum value of the product is set as the depth related value of the depth effective pixel.

In one example, the sum of the estimated depth value and the depth-related value of the effective depth pixel is the depth value after updating of the effective depth pixel.

In another example, the third product is obtained by determining the product of the depth estimate value of the depth effective pixel and the third predetermined coefficient, the product of the depth-related value and the fourth predetermined coefficient is determined to obtain the fourth product, and the sum of the third product and the fourth product is obtained Depth It is set as the depth value after the effective pixel is updated. In some embodiments, the sum of the third predetermined coefficient and the fourth predetermined coefficient is one.

In some embodiments, the depth value of the non-depth effective pixel in the second depth map is the same as the depth value of the non-depth effective pixel in the first depth map.

In some other embodiments, in order to obtain a more accurate second depth map, the depth value of the non-depth effective pixel may be updated. Thereby, the accuracy of biometric detection can be further improved.

According to the embodiment of the present invention, a Bluetooth device of a predetermined identifier can be searched for by a Bluetooth module mounted on a vehicle and a Bluetooth device of a predetermined identifier can be searched. Establish, according to the success of Bluetooth pairing/connection, wake up and control the image acquisition module mounted on the vehicle to acquire the first image of the target object, perform face recognition based on the first image, and face recognition success By sending a door unlocking command and/or door opening command to one or more doors of the vehicle according to the above, when Bluetooth pairing/connection with a Bluetooth device of a predetermined identifier is not established, the face recognition module maintains operation with low power consumption In order to do this, it can be put into a sleep state, power consumption of the unlock method by face recognition can be reduced, and the face recognition module can be operated before a user who has a Bluetooth device with a predetermined identifier arrives at the door When a user with a Bluetooth device of a predetermined identifier arrives at the door, the first image is acquired by the image acquisition module, and then the face image is processed quickly by the wake-up face recognition module In addition, it is possible to increase the efficiency of face recognition and improve the user experience. Therefore, the embodiment of the present invention can satisfy both operation with low power consumption and quick release. Employing an embodiment of the present invention, when the vehicle owner is approaching the vehicle, it is possible to automatically trigger the biometric detection and facial authentication flow without intentional action (e.g., a touch or gesture of a button), If the vehicle owner's biometric detection and face authentication are successful, the door unlocks automatically.

In one possible implementation form, the method further includes performing face recognition based on the first image and then activating a password unlocking module mounted on the vehicle according to the failure of the face recognition to start the unlocking flow by the password. include

In this realization embodiment, unlocking by password is an alternative means to unlocking by face recognition. The cause of the failure of the face recognition is that the biometric detection result indicating that the target object is non-living, that the face recognition has failed, that the image acquisition has failed (for example, the camera is broken), and that the number of recognition exceeds a predetermined number of times. One or more of these may be included. When the face recognition of the target object fails, a password unlocking flow is started. For example, the password input from the user may be acquired by the touch screen of the B pillar. In one example, if the password input is erroneous M times, for example, 5 times in a row, unlocking by the password may become invalid.

In one possible realization, the method comprises performing registration of the vehicle owner on the basis of the facial image of the vehicle owner acquired by the image acquisition module, and remote registration based on the facial image of the vehicle owner acquired by the terminal device of the vehicle owner. and sending, to the vehicle, registration information including the facial image of the vehicle owner.

In one example, performing registration of the vehicle owner based on the vehicle owner's face image acquired by the image acquisition module, when detecting that the registration button of the touch screen is clicked, requests the user to input a password, and when verification of the password is successful, Turn on the RGB camera of the image acquisition module to acquire a face image of the user, perform registration based on the acquired face image, and register facial features in this face image in advance to perform face combination upon subsequent face authentication and extracting it as a facial feature.

In one example, remote registration is performed based on the vehicle owner's face image acquired by the vehicle owner's terminal device, and registration information including the vehicle owner's face image is transmitted to the vehicle. In this example, the vehicle owner sends a registration request, which can include the vehicle owner's face image, to the TSP (Telematics Service, Provider) cloud through an App on the mobile phone, and the TSP cloud sends a registration request to the vehicle-mounted T-Box (Telematics Box) of the door unlocking device, and the vehicle T-Box activates the face recognition function according to the registration request, and the face in the face image included in the registration request The features may be pre-registered facial features and may be used for face combination in subsequent face recognition.

12 is another flowchart of a method for unlocking a door lock of a vehicle according to an embodiment of the present invention. This vehicle door unlocking method may be executed by a vehicle door unlocking device. In one possible implementation form, the method for unlocking the door of the vehicle may be realized by a processor calling out a computer readable instruction stored in a memory. In order to simplify the description, redundant description of parts similar to those described above will be omitted. As shown in Fig. 12, this vehicle door unlocking method includes steps S21 to S24.

In step 21, a Bluetooth device of a predetermined identifier is searched for by a Bluetooth module mounted in the vehicle.

In one possible implementation form, searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle is performed to a Bluetooth module mounted on the vehicle when the vehicle is in an ignition-off state or an ignition-off state and in a door unlocked state. and searching for a Bluetooth device of a predetermined identifier by

In step S22, as the Bluetooth device of the predetermined identifier can be searched for, the image acquisition module mounted on the vehicle is woken up and controlled to acquire a first image of the target object.

In one possible implementation, the number of Bluetooth devices of the given identifier is one.

In another possible implementation form, the number of Bluetooth devices of the predetermined identifier is plural, and the image acquisition module mounted on the vehicle is woken up to acquire a first image of a target object according to being able to search for a Bluetooth device of the predetermined identifier. Up-controlling includes waking up and controlling an image acquisition module mounted on the vehicle to acquire a first image of a target object in response to being able to search for any one of the Bluetooth devices of a predetermined identifier.

In one possible implementation form, waking up and controlling an image acquisition module mounted on the vehicle to acquire a first image of a target object includes waking up a face recognition module mounted on the vehicle, and the wakeup face recognition module and controlling the image acquisition module to acquire a first image of the target object by

In an embodiment of the present invention, by employing Bluetooth, it is possible to cope with long-distance communication compared to short-range sensor technologies such as ultrasonic waves or infrared rays. According to practice, the time taken for a user carrying a Bluetooth device of a predetermined identifier to arrive in the vehicle through this distance (the distance between the user and the vehicle when the Bluetooth module of the vehicle searches for the Bluetooth device of the predetermined identifier of the user) , shows that the time for the vehicle to wake up to switch the facial recognition module from the sleep state to the wake state approximately coincides. Accordingly, instead of waiting for the wake-up of the face recognition module after the user arrives at the door, when the user arrives at the door, unlocking by face recognition can be immediately performed by the wake-up face recognition module, improving the efficiency of face recognition and improve the user experience. In addition, since the user does not feel the search by Bluetooth, the user experience can be further improved. Therefore, in the embodiment of the present invention, the balance of each aspect such as power saving, user experience and security of the face recognition module is balanced by the method of waking up the face recognition module according to the ability to search for a Bluetooth device with a predetermined identifier. We provide solutions that can be taken well.

In one possible implementation form, the method further includes controlling the face recognition module to enter a sleep state when a face image cannot be acquired within a preset time after waking up the face recognition module mounted on the vehicle do.

In one possible implementation form, the method further comprises: after waking up the face recognition module mounted on the vehicle, and controlling the face recognition module to enter a sleep state when the face recognition is not successful within a preset time .

In step S23, face recognition is performed based on the first image.

In step S24, a door unlock command and/or a door open command is sent to one or more doors of the vehicle according to the success of face recognition.

In one possible implementation, sending a door unlock command and/or a door open command to one or more doors of the vehicle upon success of face recognition determines which door is authorized to unlock on the target upon success of face recognition. and sending a door unlocking command and/or a door opening command to one or more doors of the vehicle, depending on the door to which the target object is granted unlocking authority.

In one possible implementation form, the face recognition includes biometric detection and face authentication, and performing face recognition based on the first image acquires the first image by an image sensor of the image acquisition module, and the first image and performing face authentication based on pre-registered facial features, acquiring a first depth map corresponding to the first image by a depth sensor of the image acquisition module, and based on the first image and the first depth map to perform biometric detection.

In one possible implementation form, performing biometric detection based on the first image and the first depth map includes: updating the first depth map based on the first image to obtain a second depth map; and determining a biometric detection result of the target object based on the second depth map.

In one possible realization, the image sensor comprises an RGB image sensor or an infrared sensor and the depth sensor comprises a binocular infrared sensor or a time-of-flight TOF sensor.

In one possible realization, the TOF sensor uses a TOF module based on the infrared wavelength range.

In one possible implementation form, updating the first depth map based on the first image to obtain a second depth map is performed by updating a depth value of an effective depth pixel in the first depth map based on the first image. and obtaining the second depth map.

In one possible implementation, updating the first depth map based on the first image to obtain a second depth map includes: depth estimates of a plurality of pixels in the first image and a plurality of pixels based on the first image determining related information indicating the degree of relevance between each other; and updating the first depth map based on the estimated depth values of the plurality of pixels and the related information to obtain a second depth map.

In one possible implementation form, updating the first depth map to obtain a second depth map based on the estimated depth values and related information of the plurality of pixels comprises: determining a depth effective pixel in the first depth map; obtaining a depth estimated value of the depth effective pixel and depth estimated values of a plurality of neighboring pixels of the depth effective pixel from among the depth estimation values of a plurality of pixels; Acquiring the degree of relevance of a plurality of peripheral pixels of a pixel, a depth estimated value of the depth effective pixel, a depth estimated value of a plurality of peripheral pixels of the depth effective pixel, and a relationship between the depth effective pixel and a peripheral pixel of the depth effective pixel and determining a depth value after updating of the depth effective pixel based on the figure.

In one possible realization, based on a depth estimate of the depth effective pixel, a depth estimate of a plurality of neighboring pixels of the depth effective pixel, and a degree of relation between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel, the depth Determining the updated depth value of the effective pixel is based on a depth estimate value of a neighboring pixel of the depth effective pixel, and a degree of relevance between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel, the depth of the depth effective pixel determining a related value; and determining a depth value after updating of the depth effective pixel based on the depth estimation value of the depth effective pixel and the depth related value.

In one possible implementation form, determining the depth-related value of the depth effective pixel based on a depth estimate value of a neighboring pixel of the depth effective pixel and a degree of relevance between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel may include: calculating a weighted sum of depth estimation values of a plurality of peripheral pixels of the depth effective pixel by using the degree of relevance between the depth effective pixel and each surrounding pixel as the weight of each of the surrounding pixels to obtain a depth related value of the depth effective pixel .

In one possible implementation, determining a depth estimate of a plurality of pixels in the first image based on the first image comprises: determining a depth estimate of a plurality of pixels in the first image based on the first image and the first depth map. Determining a depth estimate of

In one possible implementation, determining the depth estimate values of a plurality of pixels in the first image based on the first image and the first depth map comprises the first image and the first depth map to a depth estimation neural network. input, processing, and obtaining depth estimates of a plurality of pixels in the first image.

In one possible realization, determining the depth estimate values of a plurality of pixels in the first image based on the first image and the first depth map is performed by performing a fusion process on the first image and the first depth map. obtaining a fusion result, and determining depth estimates of a plurality of pixels in the first image based on the fusion result.

In one possible realization, determining the relevant information of a plurality of pixels in the first image based on the first image is processed by inputting the first image into a relevance detection neural network, and and obtaining related information of the plurality of pixels.

In one possible implementation form, updating the first depth map based on the first image includes acquiring an image of a target object from the first image, and generating the first depth map based on the image of the target object. including updating.

In one possible implementation form, acquiring the image of the target object in the first image includes acquiring information of key points of the target object in the first image, and based on the information of key points of the target object, the and acquiring an image of the target object in a first image.

In one possible implementation form, obtaining keypoint information of the target object in the first image includes performing target detection on the first image to obtain a location area of the target object, and obtaining the target object's material area image and performing keypoint detection for the target object to obtain information of keypoints of the target object in the first image.

In one possible implementation form, updating the first depth map based on the first image to obtain a second depth map includes acquiring a depth map of the target object from the first depth map, and based on the first image and updating the depth map of the target object to obtain the second depth map.

In one possible implementation form, determining the biometric detection result of the target object based on the first image and the second depth map includes inputting the first image and the second depth map into a biometric detection neural network and processing; and obtaining a biometric detection result of the target object.

In one possible implementation form, determining the biometric detection result of the target object based on the first image and the second depth map includes performing feature extraction processing on the first image to obtain first feature information; performing feature extraction processing on the second depth map to obtain second feature information; and determining a biometric detection result of the target object based on the first feature information and the second feature information.

In one possible implementation form, determining the biometric detection result of the target object based on the first characteristic information and the second characteristic information performs a fusion process on the first characteristic information and the second characteristic information, obtaining characteristic information; and determining a biometric detection result of the target object based on the third characteristic information.

In one possible implementation form, determining the biometric detection result of the target object based on the third characteristic information includes obtaining a probability that the target object is a living body based on the third characteristic information, and a probability that the target object is a living body and determining a biometric detection result of the target object based on the .

In one possible implementation form, the method includes, after performing face recognition based on the first image, activating a password unlocking module mounted on the vehicle to initiate a flow of unlocking by a password according to a failure of face recognition. additionally include

In one possible embodiment, the method comprises performing registration of a vehicle owner based on a facial image of the vehicle owner acquired by the image acquisition module, and based on the facial image of the vehicle owner acquired by the terminal device of the vehicle owner. to perform remote registration, and to transmit registration information including a face image of the vehicle owner to the vehicle.

It should be understood that the embodiments of each method mentioned in the present invention can be combined with each other to form embodiments as long as the principles and logic are not violated. do.

For those skilled in the art, the order of each step described in the method of the specific embodiment does not strictly limit the order of implementation. It should be understood that the specific execution sequence of each step is determined by the function of each step and the possible implicit logic.

In addition, the present invention further provides a vehicle door unlocking device, an electronic device, a computer readable storage medium, and a program, all of which can be used to realize any one of the vehicle door unlocking methods provided in the present invention And, for the corresponding technical means and description, reference may be made to the description corresponding to the method, and it is omitted here.

13 is a block diagram of a door unlocking device for a vehicle according to an embodiment of the present invention. As shown in FIG. 13 , the vehicle door unlocking device can search for a Bluetooth device of a predetermined identifier and a search module 31 for searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on the vehicle. to establish a Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier, and wake up an image acquisition module mounted on the vehicle to acquire a first image of a target object according to the success of the Bluetooth pairing/connection. A wakeup module 32 for waking up and controlling an image acquisition module mounted on the vehicle to acquire a first image of a target object according to being able to control it up or search for a Bluetooth device of the predetermined identifier and a face recognition module 33 for performing face recognition based on the first image, and for sending a door unlock command and/or a door open command to one or more doors of the vehicle upon success of face recognition and an unlocking module (34).

In an embodiment of the present invention, as a Bluetooth device of a predetermined identifier can be searched for, Bluetooth pairing/connection is established between the Bluetooth module and a Bluetooth device of a predetermined identifier, and the face recognition module wakes up according to the success of Bluetooth pairing/connection. Thus, by controlling the image acquisition module to acquire the first image of the target object, the wakeup error probability of the facial recognition module can be effectively lowered by the method of waking up the face recognition module after successful Bluetooth pairing/connection, , it is possible to improve the user experience and effectively reduce the power consumption of the face recognition module.

In one possible realization, the search module 31 searches for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on the vehicle when the vehicle is in an ignition-off state or an ignition-off state and in a door lock state. In this embodiment, it is not necessary to search for the Bluetooth device of a predetermined identifier by the Bluetooth module before the vehicle's ignition is off, or before the vehicle's ignition off and when the vehicle is in the ignition off state but the door is not locked, the Bluetooth module Since there is no need to search for a Bluetooth device with a predetermined identifier, power consumption can be further reduced.

In one possible implementation, the number of Bluetooth devices of the given identifier is one.

In one possible embodiment, the number of Bluetooth devices of the given identifier is plural.

When the wakeup module 32 can search for any one of the Bluetooth devices of the predetermined identifier, it establishes Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier, or a first image of the target object. It is used to wake up and control the image acquisition module mounted on the vehicle to acquire .

In one possible implementation form, the wake-up module 32 includes a wake-up sub-module for waking up a face recognition module mounted on the vehicle, and a first image of a target object by the wake-up face recognition module. and a control sub-module for controlling the image acquisition module.

In an embodiment of the present invention, when a Bluetooth device of a predetermined identifier can be searched for, it is disclosed that a user (eg, a vehicle owner) possessing a Bluetooth device of a predetermined identifier is highly likely to enter the search range of the Bluetooth module. have. At this time, as the Bluetooth device of the predetermined identifier can be searched, Bluetooth pairing/connection is established between the Bluetooth module and the Bluetooth device of the predetermined identifier, and the face recognition module is woken up according to the success of the Bluetooth pairing/connection to the target object. By controlling the image acquisition module to acquire the first image of improved, and the power consumption of the face recognition module can be effectively reduced. In addition, the Bluetooth pairing/connection method has advantages in that it has high security and supports long-distance communication compared to short-range sensor technologies such as ultrasonic waves and infrared rays. According to practice, the time it takes for a user who has a Bluetooth device with a predetermined identifier to arrive at the vehicle through this distance (the distance between the user and the vehicle when Bluetooth pairing/connection is successful), and the vehicle sleeps the face recognition module The time to wake up to transition from state to active state is shown to be approximately coincident. Accordingly, instead of waiting for the wake-up of the face recognition module after the user arrives at the door, when the user arrives at the door, unlocking by face recognition can be immediately performed by the wake-up face recognition module, and face recognition efficiency is improved. and improve the user experience. In addition, since the user does not feel Bluetooth pairing/connection, the user experience can be further improved. Therefore, in the embodiment of the present invention, by the method of waking up the face recognition module after Bluetooth pairing/connection is successful, it is possible to achieve a good balance in each aspect such as power saving, user experience and security of the face recognition module. providing a solution.

In one possible realization, the device further comprises a first control module for controlling the face recognition module to go to sleep when the face image cannot be acquired within a preset time. This embodiment can reduce power consumption by controlling the face recognition module to enter a sleep state when a face image cannot be acquired within a preset time after waking up the face recognition module.

In one possible realization, the device further includes a second control module for controlling the face recognition module to enter a sleep state when the face recognition is not successful within a preset time. This realization mode can reduce power consumption by controlling the face recognition module to enter a sleep state when face recognition is not successful within a preset time after waking up the face recognition module.

In one possible realization, the unlocking module 34 determines a door to which unlocking authority is granted to the target object according to success of face recognition, and according to a door to which unlocking authority is granted to the target object, Used to send an unlock command and/or a door open command to one or more doors.

In one possible realization, the face recognition includes biometric detection and face authentication, and the face recognition module 33 acquires the first image by means of an image sensor of the image acquisition module, and registers the first image and the face in advance. a face authentication module for performing face authentication based on the obtained facial features; and a first depth map corresponding to the first image is acquired by a depth sensor of the image acquisition module, the first image and the first depth It has a biometric detection module for performing biometric detection based on the map.

In this embodiment, the biometric detection is for verifying whether the target object is a living body, for example, it can verify whether the target object is a human body. Facial authentication is for extracting facial features from an acquired image and combining the facial features from the acquired image with pre-registered facial features to determine whether they are facial features of the same person, for example, It may be determined whether the facial feature in the image is the facial feature of the vehicle owner.

In one possible implementation form, the biometric detection module includes an update submodule configured to update the first depth map based on the first image to obtain a second depth map, and based on the first image and the second depth map, , has a determination sub-module for determining a biometric detection result of the target object.

In one possible realization, the image sensor comprises an RGB image sensor or an infrared sensor and the depth sensor comprises a binocular infrared sensor or a time-of-flight TOF sensor. By performing the biometric detection using the depth map including the target object, depth information of the target object can be sufficiently obtained, and the accuracy of the biometric detection can be increased. In the embodiment of the present invention, for example, when the target object is a face, by performing biometric detection using a depth map including the face, depth information of face data can be sufficiently found, and the accuracy of biometric face detection can be improved. can

In one possible realization, the TOF sensor uses a TOF module based on the infrared wavelength range. By using the TOF module based on the infrared wavelength range, it is possible to reduce the influence of the external light on the imaging of the depth map.

In one possible implementation, the updating submodule is used to update the depth value of the effective depth pixel in the first depth map based on the first image to obtain the second depth map.

Here, the effective depth pixel in the depth map may refer to a pixel included in the depth map and having an invalid depth value, that is, a pixel having an incorrect depth value or a depth value that does not clearly match realistically. The number of depth effective pixels may be one or plural. By updating the depth values of one or more effective depth pixels in the depth map, the depth values of the effective depth pixels become more accurate, contributing to the improvement of the accuracy of biometric detection.

In one possible implementation form, the updating submodule determines, based on the first image, depth estimation values of a plurality of pixels in the first image and related information indicating a degree of relevance between the plurality of pixels, and the plurality of pixels is used to obtain a second depth map by updating the first depth map based on the depth estimation value and related information.

In one possible embodiment, the updating submodule is configured to determine the effective depth pixel in the first depth map, and, among the estimated depth values of the plurality of pixels, the estimated depth of the depth effective pixel and a plurality of perimeters of the depth effective pixel. obtaining a depth estimate value of a pixel; obtaining, from the related information of the plurality of pixels, a degree of relation between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel; a depth estimate value of the depth effective pixel; used to determine a depth value after updating of the depth effective pixel based on the estimated depth values of the plurality of peripheral pixels of the depth effective pixel and the degree of relation between the depth effective pixel and the peripheral pixels of the depth effective pixel.

In one possible implementation mode, the updating submodule is configured to calculate a depth-related value of the depth effective pixel based on a depth estimate value of a neighboring pixel of the depth effective pixel and a degree of relation between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel. and determining the updated depth value of the depth effective pixel based on the depth estimation value of the depth effective pixel and the depth-related value.

In one possible implementation mode, the update submodule calculates a weighted sum of depth estimation values of a plurality of neighboring pixels of the depth effective pixel by using the degree of relevance between the depth effective pixel and each neighboring pixel as a weight of each neighboring pixel to calculate the depth. It is used to obtain the depth-related value of the effective pixel.

In one possible realization, the updating sub-module is used to determine depth estimates of a plurality of pixels in the first image based on the first image and the first depth map.

In one possible implementation, the update submodule is used to input the first image and the first depth map to a depth estimation neural network for processing, and to obtain depth estimates of a plurality of pixels in the first image.

In a possible implementation mode, the update submodule performs fusion processing on the first image and the first depth map to obtain a fusion result, and the depth of a plurality of pixels in the first image based on the fusion result It is used to determine the estimate.

In one possible realization, the update submodule is used to input the first image into a relevance detection neural network for processing, and to obtain relevant information of a plurality of pixels in the first image.

In one possible realization, the update sub-module is used to acquire an image of the target object from the first image, and to update the first depth map based on the image of the target object.

In one possible implementation form, the update submodule is configured to acquire information of keypoints of the target object in the first image, and acquire the image of the target object in the first image based on the keypoint information of the target object. used

In one example, an outline of the target object is determined based on information of key points of the target object, and an image of the target object is cut out from the first image based on the outline of the target object. Compared with the positional information of the target obtained by target detection, since the position of the target obtained by the information of the keypoint is more accurate, it contributes to the improvement of the accuracy of subsequent biometric detection.

In this way, by acquiring the image of the target object from the first image and performing biometric detection based on the image of the target object, it is possible to reduce the interference with the biometric detection by the background information in the first image.

In one possible realization, the update submodule performs target detection on a first image to obtain a location area of the target object, and performs keypoint detection on an image of a material area of the target object, so as to It is used to obtain key point information of the target object.

In one possible implementation form, the update submodule acquires the depth map of the target object from the first depth map, and updates the depth map of the target object based on the first image to obtain the second depth map used for

In this way, by obtaining a depth map of the target object from the first depth map, updating the depth map of the target object based on the first image to obtain a second depth map, biometric detection using the background information of the first depth map interference can be reduced.

In some specific scenes (eg, a scene with strong outdoor light), a partial area of an acquired depth map (eg, a depth map acquired by a depth sensor) may become effective. In addition, even in normal light irradiation, partial ineffectiveness of the depth map is randomly caused by factors such as reflection by glasses, dark hair, or black glasses frame. Some special papers also cause large area or partial ineffectiveness in depth maps of similarly printed face photos. Also, when the active light source of the depth sensor is blocked, there may be a case in which a part of the depth map is invalid, but the image formation in the non-living image sensor becomes normal. In this way, when a part or all of the depth map becomes invalid, an error occurs in the identification of a living body and a non-living body based on the depth map. Accordingly, in the embodiment of the present invention, by restoring or updating the first depth map and performing biodetection using the repaired or updated depth map, it contributes to the improvement of the accuracy of biometric detection.

In one possible implementation, the determining submodule is used to input the first image and the second depth map into a biometric detection neural network for processing, and to obtain a biometric detection result of the target object.

In one possible implementation mode, the determining submodule performs feature extraction processing on the first image to obtain first feature information, and performs feature extraction processing on the second depth map to obtain second feature information; It is used to determine a biometric detection result of the target object based on the first characteristic information and the second characteristic information.

Here, optionally, the feature extraction processing can be realized by a neural network or other machine learning algorithm, and the type of the extracted feature information can optionally be obtained by learning using a sample, but in an embodiment of the present invention does not limit them.

In one possible implementation mode, the determining submodule performs fusion processing on the first characteristic information and the second characteristic information to obtain third characteristic information, and based on the third characteristic information, the living body of the target object It is used to determine the detection result.

In one possible realization, the determining submodule is configured to obtain a probability that the target object is a living body based on the third characteristic information, and to determine a biometric detection result of the target object based on a probability that the target object is a living body used

In one possible realization, the device further comprises an activation triggering module for activating a password unlocking module mounted on the vehicle to trigger a password unlocking flow according to a face recognition failure.

In this realization embodiment, unlocking by password is an alternative means to unlocking by face recognition. Causes of face recognition failure include biometric detection results indicating that the target object is non-living, face recognition failure, image acquisition failure (e.g., camera malfunction), and recognition times exceeding a predetermined number of times. One or more of these may be included. When the face recognition of the target object fails, the flow of unlocking by the password is started. For example, you may make it acquire the password input from a user by the touch screen of a B-pillar.

In one possible realization, the device registers the vehicle owner based on the vehicle owner's face image acquired by the image acquisition module, and based on the vehicle owner's face image acquired by the vehicle owner's terminal device and a registration module used for one or two of performing remote registration by doing so and transmitting registration information including a face image of the vehicle owner to the vehicle.

According to this realization mode, at the time of subsequent face authentication, face combination can be performed based on this pre-registered facial feature.

In some embodiments, functions or modules provided in the apparatus provided in the embodiments of the present invention may be used to execute the methods described in the method embodiments, and for specific implementation thereof, refer to the description of the method embodiments, and description For the sake of simplicity, redundant descriptions are omitted here.

14 is a block diagram of an in-vehicle face authentication unlock system according to an embodiment of the present invention. 14 , this vehicle-mounted face authentication unlocking system includes a memory 41 , a face recognition module 42 , an image acquisition module 43 , and a Bluetooth module 44 , and the face recognition module 42 is connected to the memory 41, the image acquisition module 43, and the Bluetooth module 44, respectively, and the Bluetooth module 44 succeeds in Bluetooth pairing/connection with a Bluetooth device of a predetermined identifier. When the Bluetooth device of the predetermined identifier can be searched for, or a microprocessor 441 for waking up the face recognition module 42, and a Bluetooth sensor 442 connected to the microprocessor 441, , a communication interface connected to the door domain controller is additionally installed in the face recognition module 42 , and when face recognition is successful, control information for unlocking the door is transmitted to the door domain controller through the communication interface.

In one example, the memory 41 may include one or more of a flash memory (Flash) and a DDR3 (Double Date Rate 3, Third Generation Double Data Rate) memory.

In one example, the face recognition module 42 may be realized using a System on Chip (SoC).

In one example, the face recognition module 42 is connected to the door domain controller via a CAN (Controller Area Network) bus.

In one example, the image acquisition module 43 includes an image sensor and a depth sensor.

In one example, the depth sensor has at least one of a binocular infrared sensor and a time-of-flight TOF sensor.

In one possible embodiment the depth sensor comprises a binocular infrared sensor in which two infrared cameras are installed on either side of the camera of the image sensor. For example, in the example shown in Fig. 4A, the image sensor is an RGB sensor, the camera of the image sensor is an RGB camera, the depth sensor is a binocular infrared sensor, has two IR (infrared) cameras, and two Infrared cameras are installed on both sides of the RGB cameras of the image sensor.

In one example, the image acquisition module 43 is installed between the infrared camera of the binocular infrared sensor and the camera of the image sensor, and further includes one or more beam lights including at least one of a beam light for an image sensor and a beam light for a depth sensor. include For example, when the image sensor is an RGB sensor, the beam light for the image sensor is a white light, when the image sensor is an infrared sensor, the beam light for the image sensor is an infrared light, and when the depth sensor is a binocular infrared sensor, the depth of field The light beam for sensors may be an infrared light. In the example shown in FIG. 4A, an infrared light is provided between the infrared camera of a binocular infrared sensor and the camera of an image sensor. For example, you may make it use infrared rays of 940 nm as an infrared light.

In one example, the light beam light may be in the normally open mode. In this example, when the camera of the image acquisition module is in the operating state, the beam light is in the on state.

In another example, when the light beam is insufficient, the light beam light may be turned on. For example, the ambient light intensity may be acquired by the ambient light sensor, and when the ambient light intensity is lower than the light intensity threshold, it may be determined that the light beam is insufficient, and the auxiliary light may be turned on.

In one possible embodiment, the image acquisition module 43 further comprises a laser installed between the camera of the depth sensor and the camera of the image sensor. For example, in the example shown in Fig. 4B, the image sensor is an RGB sensor, the image sensor camera is an RGB camera, the depth sensor is a TOF sensor, and a laser is provided between the TOF sensor camera and the RGB sensor camera. For example, the laser may be a VCSEL, and the TOF sensor may acquire a depth map with laser light emitted from the VCSEL.

In one example, the depth sensor is connected to the face recognition module 42 through an LVDS (Low-Voltage Differential Signaling) interface.

In one possible realization, the vehicle-mounted face authentication unlocking system is connected to the face recognition module 42 and further includes a password unlocking module 45 for unlocking the door lock.

In one possible implementation, the password unlocking module 45 has one or two of a touch screen and a keyboard.

In one example, the touch screen is connected to the face recognition module 42 via a Flat Panel Display Link (FPD-Link).

In one possible realization, the vehicle-mounted face authentication unlocking system further includes a battery module 46 respectively connected to the microprocessor 441 and the face recognition module 42 .

In one possible implementation form, the memory 41 , the face recognition module 42 , the Bluetooth module 44 , and the battery module 46 may be mounted on an ECU (Electronic Control Unit).

15 is a schematic diagram of an in-vehicle face authentication unlocking system according to an embodiment of the present invention. In the example shown in Fig. 15, the face recognition module is realized using the SoC 101, the memory has a flash memory (Flash) 102 and a DDR3 memory 103, and the Bluetooth module is a Bluetooth sensor 104 and a microprocessor. (MCU, Microcontroller Unit 105, SoC 101, flash memory 102, DDR3 memory 103, Bluetooth sensor 104, microprocessor 105 and battery module 106 are ECU 100 The image acquisition module has a depth sensor 200 connected to the SoC 101 via an LVDS interface, and the password unlocking module is a touch screen 300 connected to the SoC 101 via FPD-Link. , and the SoC 101 is connected to the door domain controller 400 through a CAN bus, Fig. 16 is a schematic diagram of a vehicle according to an embodiment of the present invention. An in-vehicle face authentication unlock system 51 connected to a domain controller 52 is provided.

In one possible realization, the image acquisition module is installed outside the vehicle cabin.

In one possible realization, the image acquisition module is installed on one or more of a B-pillar, one or more doors, and one or more rearview mirrors of the vehicle.

In one possible embodiment, the face recognition module is mounted in the vehicle and is connected to the door domain controller via a CAN bus.

An embodiment of the present invention is a computer program comprising computer readable code, wherein the computer readable code further includes a computer program that, when operated in an electronic device, causes a processor of the electronic device to execute instructions for realizing the method. to provide.

An embodiment of the present invention further provides a computer-readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the computer-readable storage medium realizes the method. A computer-readable storage medium may be a nonvolatile computer-readable storage medium, or a volatile computer-readable storage medium may be sufficient as it.

An embodiment of the present invention further provides an electronic device comprising a processor and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the method.

The electronic device may be provided as a terminal, server, or other type of device.

Fig. 17 shows a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a message transmitting/receiving device, a game console, a tablet type device, a medical device, a fitness device, a terminal such as a personal digital assistant.

Referring to FIG. 17 , the electronic device 800 includes a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , and an input/output (I/O) interface. 812 , a sensor component 814 , and a communication component 816 .

The processing component 802 typically controls the overall operation of the electronic device 800 , such as operations related to display, phone call, data communication, camera operation, and recording operation. The processing component 802 may include one or more processors 820 executing instructions to carry out all or some steps of the method. Further, processing component 802 may include one or more modules for interaction with other components. For example, processing component 802 may include a multimedia module for interaction with multimedia component 808 .

The memory 804 is configured to store various types of data to support operation in the electronic device 800 . These data include, for example, commands of various application programs or methods operated by the electronic device 800 , contact data, phone book data, messages, pictures, videos, and the like. Memory 804 may include, for example, static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory ( ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk, etc., can be realized by various types of volatile or nonvolatile storage devices, or combinations thereof.

The power component 806 supplies power to each component of the electronic device 800 . Power component 806 may include a power management system, one or more power sources, and other components related to power generation, management, and distribution for electronic device 800 .

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, it may be realized as a touch screen that receives an input signal from a user. The touch panel includes one or more touch sensors to detect touches, slides and gestures on the touch panel. The touch sensor may not only detect a boundary of a touch or slide operation, but also detect a duration and pressure associated with the touch or slide operation. In some embodiments, multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, for example, a photographing mode or an imaging mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may have a fixed optical lens system or focal length and optical zoom capability.

The audio component 810 is configured to output and/or input an audio signal. For example, the audio component 810 includes one microphone MIC, and the microphone MIC is when the electronic device 800 is in an operation mode, for example, a call mode, a recording mode, and a voice recognition mode. , configured to receive an external audio signal. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, the audio component 810 further includes a speaker for outputting an audio signal.

I/O interface 812 provides an interface between processing component 802 and a peripheral interface module, which may be a keyboard, click wheel, button, or the like. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

The sensor component 814 includes one or more sensors for status evaluation at each side of the electronic device 800 . For example, the sensor component 814 may detect an on/off state of the electronic device 800 , eg, a relative positioning of components such as a display device and a keypad of the electronic device 800 , the sensor component 814 further indicates a change in the position of the electronic device 800 or a component in which the electronic device 800 is located, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration/deceleration of the electronic device 800, and the electronic device 800 ) can be detected. The sensor component 814 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor component 814 may further include a photosensor for use in imaging applications, such as CMOS or CCD image sensors. In some embodiments, the sensor component 814 may further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to realize wired or wireless communication between the electronic device 800 and another device. The electronic device 800 may access a wireless network based on a communication standard, for example, WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system through a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate near field communication. For example, the NFC module may be realized by radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, the electronic device 800 includes one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs). , implemented by a controller, microcontroller, microprocessor or other electronic element, and may be used to carry out the method.

In the exemplary embodiment, there is further provided a nonvolatile computer readable storage medium, for example, a memory 804 containing computer program instructions, which are executed by the processor 820 of the electronic device 800 . Once executed, the method may be executed.

The invention may be a system, method and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions for realizing each aspect of the present invention in the processor.

The computer-readable storage medium may be a tangible device capable of storing and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electronic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory); Static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, e.g., punched cards or slots in which instructions are stored; such mechanical encoding devices, and any suitable combination of the above. As used herein, a computer-readable storage medium is an instantaneous signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating via waveguides or other transmission media (eg, optical pulses passing through an optical fiber cable). ), or an electrical signal transmitted via a wire.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing/processing device, or may be downloaded to an external computer via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. Alternatively, it may be downloaded to an external storage device. The network may include copper transport cables, fiber optic transport, wireless transport, routers, firewalls, switchboards, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives a computer readable program command from the network, transmits the computer readable program command, and stores the computer readable program command in a computer readable storage medium in each computing/processing device.

The computer program instructions for carrying out the operations of the present invention may include assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine dependent instructions, microcode, firmware instructions, state setting data or an object-oriented programming language such as Smalltalk, C++, etc.; and source code or target code written in any combination of one or more programming languages including a general procedural programming language such as "C" language or a similar programming language. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partly on a remote computer, or It may run entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any kind of network, including a local area network (LAN) or a wide area network (WAN), or (eg, an Internet service provider via the Internet) may be connected to an external computer. In some embodiments, state information from computer readable program instructions is used to customize electronic circuitry, such as, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuitry. Each aspect of the present invention may be realized by executing computer readable program instructions by

Although each aspect of the present invention has been described herein with reference to a flowchart and/or block diagram of a method, apparatus (system) and computer program product according to an embodiment of the present invention, each block in the flowchart and/or block diagram, and the flowchart and It should be understood that all combinations of blocks in the block diagram may be realized by computer readable program instructions.

These computer readable program instructions are provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device, and when these instructions are executed by the processor of the computer or other programmable data processing device, one or more of the flowcharts and/or block diagrams The device may be manufactured to realize the functions/actions specified in the block. These computer readable program instructions may be stored in a computer readable storage medium, and may cause a computer, a programmable data processing apparatus, and/or other apparatus to operate in a specific manner. A computer-readable storage medium having instructions stored thereon includes a product having instructions for realizing each aspect of a function/action specified in one or more blocks of a flowchart and/or a block diagram.

The computer readable program instructions are loaded into a computer, other programmable data processing device, or other device, and cause the computer, other programmable data processing device or other device to execute a series of operational steps, thereby creating a process executed by the computer; , other programmable data processing device or other device to realize the specified functions/operations in one or more blocks of the flowchart and/or block diagram by instructions executed on the device.

Flowcharts and block diagrams in the drawings represent realizable system architectures, functions, and operations of systems, methods, and computer program products according to a plurality of embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a single module, program segment, or portion of an instruction, wherein the module, program segment or portion of the instruction includes one or more executable instructions for realizing a specified logical function. includes In some alternative implementations, the functions indicated in the blocks may be realized in a different order from the order in the drawings. For example, two consecutive blocks may be executed substantially simultaneously, or may be executed in the reverse order depending on the function involved. In addition, each block in the block diagram and/or flowchart and the combination of blocks in the block diagram and/or flowchart may be realized by a dedicated system based on hardware for executing designated functions or operations, or dedicated hardware and a computer It should also be noted that it may be realized by a combination of commands.

As mentioned above, although each embodiment of this invention was described, the said description is only exemplary, and is not exhaustive, and is not limited to each disclosed embodiment, either. Various modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of each described embodiment. The terminology selected herein is intended to preferably interpret the principle, practical application, or improvement of existing techniques of each embodiment, or to enable others skilled in the art to understand each embodiment disclosed herein.

Claims (108)

Searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle;
establishing Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier as the Bluetooth device of the predetermined identifier can be searched;
waking up and controlling the image acquisition module mounted on the vehicle to acquire a first image of the target object according to the success of the Bluetooth pairing/connection;
performing face recognition based on the first image;
and sending a door unlock command and/or a door open command to one or more doors of the vehicle according to the success of facial recognition. The method of claim 1,
Searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle includes:
and when the vehicle is in an ignition-off state or an ignition-off state and in a door lock state, searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted in the vehicle. 3. The method according to claim 1 or 2,
and the number of Bluetooth devices of the predetermined identifier is one. 3. The method according to claim 1 or 2,
The number of Bluetooth devices of the predetermined identifier is plural,
Establishing Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier as the Bluetooth device of the predetermined identifier can be searched for,
A method for unlocking a vehicle door, comprising: establishing Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier in response to being able to search for any one of the Bluetooth devices of the predetermined identifier. 5. The method according to any one of claims 1 to 4,
Wake up and control the image acquisition module mounted on the vehicle to acquire the first image of the target object,
waking up the face recognition module mounted on the vehicle;
and controlling the image acquisition module to acquire a first image of a target object by the wake-up face recognition module. 6. The method of claim 5,
After waking up the face recognition module mounted on the vehicle,
The method of claim 1, further comprising controlling the face recognition module to enter a sleep state when a face image cannot be acquired within a preset time. 6. The method of claim 5,
After waking up the face recognition module mounted on the vehicle,
The method of claim 1, further comprising controlling the face recognition module to enter a sleep state when face recognition is not successful within a preset time. 8. The method according to any one of claims 1 to 7,
Sending a door unlock command and/or a door open command to one or more doors of the vehicle upon success of facial recognition comprises:
determining a door to which unlocking authority is granted to the target object according to the success of face recognition;
and sending a door unlocking command and/or a door opening command to one or more doors of the vehicle according to a door for which unlocking authority is granted to the target object. 9. The method according to any one of claims 1 to 8,
The face recognition includes biometric detection and face authentication,
Performing face recognition based on the first image comprises:
acquiring the first image by an image sensor of the image acquisition module, and performing face authentication based on the first image and pre-registered facial features;
acquiring a first depth map corresponding to the first image by a depth sensor of the image acquisition module, and performing biometric detection based on the first image and the first depth map; Way. 10. The method of claim 9,
Performing biometric detection based on the first image and the first depth map,
updating the first depth map based on the first image to obtain a second depth map;
and determining a biometric detection result of the target object based on the first image and the second depth map. 11. The method according to claim 9 or 10,
The image sensor includes an RGB image sensor or an infrared sensor,
The depth sensor comprises a binocular infrared sensor or a time-of-flight TOF sensor. 12. The method of claim 11,
The TOF sensor uses a TOF module based on an infrared wavelength region, the method of unlocking a door of a vehicle. 13. The method according to any one of claims 10 to 12,
To obtain a second depth map by updating the first depth map based on the first image,
and obtaining the second depth map by updating a depth value of an effective depth pixel in the first depth map based on the first image. 14. The method according to any one of claims 10 to 13,
To obtain a second depth map by updating the first depth map based on the first image,
determining, based on the first image, depth estimation values of a plurality of pixels in the first image and related information indicating a degree of relevance between the plurality of pixels;
and updating the first depth map based on the depth estimation values of the plurality of pixels and related information to obtain a second depth map. 15. The method of claim 14,
obtaining a second depth map by updating the first depth map based on the depth estimation values of the plurality of pixels and related information;
determining an effective depth pixel in the first depth map;
obtaining, from among the depth estimation values of the plurality of pixels, a depth estimation value of the depth effective pixel and depth estimation values of a plurality of neighboring pixels of the depth effective pixel;
acquiring a degree of relevance between the depth effective pixel and a plurality of peripheral pixels of the depth effective pixel from the related information of the plurality of pixels;
Based on the estimated depth value of the depth effective pixel, the estimated depth values of a plurality of neighboring pixels of the depth effective pixel, and the degree of relevance between the depth effective pixel and the neighboring pixels of the depth effective pixel, the updated depth value of the depth effective pixel is determined A method of unlocking a door of a vehicle, comprising determining. 16. The method of claim 15,
The depth after updating of the depth effective pixel is based on the estimated depth of the depth effective pixel, the estimated depth of the plurality of peripheral pixels of the depth effective pixel, and the degree of relevance between the depth effective pixel and the plurality of peripheral pixels of the depth effective pixel Determining the value is
determining a depth-related value of the depth effective pixel based on a depth estimate value of a neighboring pixel of the depth effective pixel and a degree of relation between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel;
determining the updated depth value of the depth effective pixel based on the estimated depth value of the depth effective pixel and the depth-related value; A method of unlocking a door of a vehicle, comprising: 17. The method of claim 16,
Determining the depth-related value of the depth effective pixel based on the estimated depth value of the neighboring pixel of the depth effective pixel and the degree of relation between the depth effective pixel and the plurality of neighboring pixels of the depth effective pixel includes:
calculating a weighted sum of depth estimates of a plurality of neighboring pixels of the depth effective pixel by using the degree of relevance between the depth effective pixel and each neighboring pixel as a weight of each neighboring pixel to obtain a depth relation value of the depth effective pixel How to unlock a vehicle door. 18. The method according to any one of claims 14 to 17,
determining, based on the first image, depth estimates of a plurality of pixels in the first image,
and determining depth estimates of a plurality of pixels in the first image based on the first image and the first depth map. 19. The method of claim 18,
Determining depth estimates of a plurality of pixels in the first image based on the first image and the first depth map comprises:
and inputting the first image and the first depth map into a depth estimation neural network for processing, and obtaining depth estimation values of a plurality of pixels in the first image. 20. The method according to claim 18 or 19,
Determining depth estimates of a plurality of pixels in the first image based on the first image and the first depth map comprises:
performing fusion processing on the first image and the first depth map to obtain a fusion result;
and determining depth estimates of a plurality of pixels in the first image based on the fusion result. 21. The method according to any one of claims 14 to 20,
Based on the first image, determining the relevant information of the plurality of pixels in the first image comprises:
The method of claim 1, comprising inputting the first image into a relevance detection neural network for processing, and obtaining related information of a plurality of pixels from the first image. 22. The method according to any one of claims 10-21,
Updating the first depth map based on the first image comprises:
acquiring an image of the target object from the first image;
and updating the first depth map based on the image of the target object. 23. The method of claim 22,
Acquiring the image of the target object from the first image comprises:
acquiring information of key points of the target object in the first image;
and acquiring an image of the target object from the first image based on the key point information of the target object. 24. The method of claim 23,
Acquiring the key point information of the target object in the first image comprises:
performing target detection on the first image to obtain a material area of the target object;
and performing keypoint detection on an image of a material region of the target object to obtain keypoint information of the target object in the first image. 25. The method according to any one of claims 10-24,
To obtain a second depth map by updating the first depth map based on the first image,
acquiring a depth map of the target object from the first depth map;
and updating a depth map of the target object based on the first image to obtain the second depth map. 26. The method according to any one of claims 10 to 25,
Determining the biometric detection result of the target object based on the first image and the second depth map,
and inputting the first image and the second depth map into a bio-detection neural network and processing the first image and the second depth map, and obtaining a bio-detection result of the target object. 27. The method according to any one of claims 10-26,
Determining the biometric detection result of the target object based on the first image and the second depth map,
performing feature extraction processing on the first image to obtain first feature information;
performing feature extraction processing on the second depth map to obtain second feature information;
and determining a biometric detection result of the target object based on the first characteristic information and the second characteristic information. 28. The method of claim 27,
Determining the biometric detection result of the target object based on the first characteristic information and the second characteristic information includes:
performing fusion processing on the first characteristic information and the second characteristic information to obtain third characteristic information;
and determining a biometric detection result of the target object based on the third characteristic information. 29. The method of claim 28,
Determining the biometric detection result of the target object based on the third characteristic information includes:
obtaining a probability that the target object is a living body based on the third characteristic information;
and determining a biometric detection result of the target object based on a probability that the target object is a living body. 30. The method according to any one of claims 1 to 29,
After face recognition is performed based on the first image,
The method of unlocking a door of a vehicle, further comprising activating a password unlocking module mounted on the vehicle to start a flow of unlocking by a password according to a face recognition failure. 31. The method according to any one of claims 1 to 30,
performing registration of the vehicle owner based on the facial image of the vehicle owner acquired by the image acquisition module;
Further comprising at least one of performing remote registration based on the face image of the vehicle owner acquired by the terminal device of the vehicle owner, and transmitting registration information including the face image of the vehicle owner to the vehicle , how to unlock the door of the vehicle. Searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle;
waking up and controlling an image acquisition module mounted on the vehicle to acquire a first image of a target object as the Bluetooth device of the predetermined identifier can be searched;
performing face recognition based on the first image;
and sending a door unlock command and/or a door open command to one or more doors of the vehicle according to the success of facial recognition. 33. The method of claim 32,
Searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle includes:
and when the vehicle is in an ignition-off state or an ignition-off state and in a door lock state, searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted in the vehicle. 33. The method of claim 31 or 32,
and the number of Bluetooth devices of the predetermined identifier is one. 33. The method of claim 31 or 32,
The number of Bluetooth devices of the predetermined identifier is plural,
Wake up and control the image acquisition module mounted on the vehicle to acquire a first image of a target object as the Bluetooth device of the predetermined identifier can be searched for,
A method for unlocking a door of a vehicle, comprising: waking up and controlling an image acquisition module mounted on the vehicle to acquire a first image of a target object according to being able to search for any one of the Bluetooth devices of a predetermined identifier. 36. The method according to any one of claims 32 to 35,
Wake up and control the image acquisition module mounted on the vehicle to acquire the first image of the target object,
waking up the face recognition module mounted on the vehicle;
and controlling an image acquisition module to acquire a first image of a target object by the wake-up face recognition module. 37. The method of claim 36,
After waking up the face recognition module mounted on the vehicle,
The method of claim 1, further comprising controlling the face recognition module to enter a sleep state when a face image cannot be acquired within a preset time. 37. The method of claim 36,
After waking up the face recognition module mounted on the vehicle,
The method of claim 1, further comprising controlling the face recognition module to enter a sleep state when face recognition is not successful within a preset time. 39. The method according to any one of claims 32 to 38,
Upon success of facial recognition, sending a door unlock command and/or a door open command to one or more doors of the vehicle comprises:
determining a door to which unlocking authority is granted to the target object according to the success of face recognition;
and sending a door unlocking command and/or a door opening command to one or more doors of the vehicle according to a door for which unlocking authority is granted to the target object. 40. The method according to any one of claims 32 to 39,
The face recognition includes biometric detection and face authentication,
Performing face recognition based on the first image comprises:
acquiring the first image by an image sensor of the image acquisition module, and performing face authentication based on the first image and pre-registered facial features;
acquiring a first depth map corresponding to the first image by a depth sensor of the image acquisition module, and performing biometric detection based on the first image and the first depth map; Way. 41. The method of claim 40,
Performing biometric detection based on the first image and the first depth map,
updating the first depth map based on the first image to obtain a second depth map;
and determining a biometric detection result of the target object based on the first image and the second depth map. 42. The method of claim 40 or 41,
The image sensor includes an RGB image sensor or an infrared sensor,
The depth sensor comprises a binocular infrared sensor or a time-of-flight TOF sensor. 43. The method of claim 42,
The TOF sensor uses a TOF module based on an infrared wavelength region, the method of unlocking a door of a vehicle. 44. The method according to any one of claims 41 to 43,
To obtain a second depth map by updating the first depth map based on the first image,
and obtaining the second depth map by updating a depth value of an effective depth pixel in the first depth map based on the first image. 45. The method according to any one of claims 41 to 44,
To obtain a second depth map by updating the first depth map based on the first image,
determining, based on the first image, depth estimation values of a plurality of pixels in the first image and related information indicating a degree of relevance between the plurality of pixels;
and updating the first depth map based on the depth estimation values of the plurality of pixels and related information to obtain a second depth map. 46. The method of claim 45,
obtaining a second depth map by updating the first depth map based on the depth estimation values of the plurality of pixels and related information;
determining an effective depth pixel in the first depth map;
obtaining, from among the depth estimation values of the plurality of pixels, a depth estimation value of the depth effective pixel and depth estimation values of a plurality of neighboring pixels of the depth effective pixel;
acquiring a degree of relevance between the depth effective pixel and a plurality of peripheral pixels of the depth effective pixel from the related information of the plurality of pixels;
Based on the estimated depth value of the depth effective pixel, the estimated depth values of a plurality of neighboring pixels of the depth effective pixel, and the degree of relevance between the depth effective pixel and the neighboring pixels of the depth effective pixel, the updated depth value of the depth effective pixel is determined A method of unlocking a door of a vehicle, comprising determining. 47. The method of claim 46,
The depth after updating of the depth effective pixel is based on the estimated depth of the depth effective pixel, the estimated depth of the plurality of peripheral pixels of the depth effective pixel, and the degree of relevance between the depth effective pixel and the plurality of peripheral pixels of the depth effective pixel Determining the value is
determining a depth-related value of the depth effective pixel based on a depth estimate value of a neighboring pixel of the depth effective pixel and a degree of relation between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel;
and determining a depth value after updating of the depth effective pixel based on the depth estimation value of the depth effective pixel and the depth-related value. 48. The method of claim 47,
Determining the depth-related value of the depth effective pixel based on the estimated depth value of the neighboring pixel of the depth effective pixel and the degree of relation between the depth effective pixel and the plurality of neighboring pixels of the depth effective pixel includes:
calculating the weighted sum of the depth estimation values of a plurality of neighboring pixels of the depth effective pixel using the degree of relevance between the depth effective pixel and each neighboring pixel as the weight of each neighboring pixel, and taking the depth relation value of the depth effective pixel How to unlock the vehicle door. 49. The method according to any one of claims 45 to 48,
determining, based on the first image, depth estimates of a plurality of pixels in the first image,
and determining depth estimates of a plurality of pixels in the first image based on the first image and the first depth map. 50. The method of claim 49,
Determining depth estimation values of a plurality of pixels in the first image based on the first image and the first depth map comprises:
and inputting the first image and the first depth map into a depth estimation neural network for processing, and obtaining depth estimation values of a plurality of pixels in the first image. 51. The method of claim 49 or 50,
Determining depth estimates of a plurality of pixels in the first image based on the first image and the first depth map comprises:
performing fusion processing on the first image and the first depth map to obtain a fusion result;
and determining depth estimates of a plurality of pixels in the first image based on the fusion result. 52. The method according to any one of claims 45 to 51,
Determining, based on the first image, related information of the plurality of pixels in the first image,
and inputting the first image into a relevance detection neural network for processing, and obtaining related information of a plurality of pixels in the first image. 53. The method according to any one of claims 41 to 52,
Updating the first depth map based on the first image comprises:
acquiring an image of the target object from the first image;
and updating the first depth map based on the image of the target object. 54. The method of claim 53,
Acquiring the image of the target object from the first image comprises:
acquiring information on key points of the target object in the first image;
and acquiring an image of the target object from the first image based on the key point information of the target object. 55. The method of claim 54,
Acquiring the key point information of the target object in the first image comprises:
performing target detection on the first image to obtain a material area of the target object;
and performing keypoint detection on an image of a material region of the target object to obtain keypoint information of the target object in the first image. 56. The method according to any one of claims 41 to 55,
To obtain a second depth map by updating the first depth map based on the first image,
acquiring a depth map of the target object from the first depth map;
and updating a depth map of the target object based on the first image to obtain the second depth map. 57. The method according to any one of claims 41 to 56,
Determining the biometric detection result of the target object based on the first image and the second depth map,
and inputting the first image and the second depth map into a bio-detection neural network and processing the first image and the second depth map, and obtaining a bio-detection result of the target object. 58. The method according to any one of claims 41 to 57,
Determining the biometric detection result of the target object based on the first image and the second depth map,
performing feature extraction processing on the first image to obtain first feature information;
performing feature extraction processing on the second depth map to obtain second feature information;
and determining a biometric detection result of the target object based on the first characteristic information and the second characteristic information. 59. The method of claim 58,
Determining the biometric detection result of the target object based on the first characteristic information and the second characteristic information includes:
performing fusion processing on the first characteristic information and the second characteristic information to obtain third characteristic information;
and determining a biometric detection result of the target object based on the third characteristic information. 60. The method of claim 59,
Determining the biometric detection result of the target object based on the third characteristic information includes:
obtaining a probability that the target object is a living body based on the third characteristic information;
and determining a biometric detection result of the target object based on a probability that the target object is a living body. 61. The method according to any one of claims 32 to 60,
After face recognition is performed based on the first image,
The method for unlocking a door of a vehicle, further comprising activating a password unlocking module mounted on the upper vehicle to start a flow of unlocking by a password according to a face recognition failure. 62. The method according to any one of claims 32 to 61,
performing registration of the vehicle owner based on the facial image of the vehicle owner acquired by the image acquisition module;
Further comprising at least one of performing remote registration based on the face image of the vehicle owner acquired by the terminal device of the vehicle owner, and transmitting registration information including the face image of the vehicle owner to the vehicle , unlocking the vehicle's door. a search module for searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on a vehicle;
As the Bluetooth device of the predetermined identifier can be searched for, a Bluetooth pairing/connection is established between the Bluetooth module and the Bluetooth device of the predetermined identifier, and a first image of the target object is acquired according to the success of the Bluetooth pairing/connection. Wake up and control the image acquisition module mounted on the vehicle to wake up the image acquisition module mounted on the vehicle, or wake up the image acquisition module mounted on the vehicle to acquire the first image of the target object according to the ability to search for the Bluetooth device of the predetermined identifier a wake-up module for controlling
a face recognition module for performing face recognition based on the first image;
and an unlocking module for sending a door unlocking command and/or a door open command to one or more doors of the vehicle upon success of facial recognition. 64. The method of claim 63,
The search module is
used for searching for a Bluetooth device of a predetermined identifier by a Bluetooth module mounted on the vehicle when the vehicle is in an ignition-off state or an ignition-off and door lock state. 65. The method of claim 63 or 64,
and the number of Bluetooth devices of the predetermined identifier is one. 65. The method of claim 63 or 64,
The number of Bluetooth devices of a given identifier is plural,
The wake-up module is
An image mounted on the vehicle to establish Bluetooth pairing/connection between the Bluetooth module and the Bluetooth device of the predetermined identifier or to acquire a first image of a target object as any one of the Bluetooth devices of the predetermined identifier can be searched A door unlocking device for a vehicle, used for waking up and controlling the acquisition module. 67. The method according to any one of claims 63 to 66,
The wake-up module is
a wake-up sub-module for waking up the facial recognition module mounted on the vehicle;
and a sub-module for controlling the image acquisition module to acquire a first image of a target object by the wake-up face recognition module. 68. The method of claim 67,
and a first control module for controlling the face recognition module to enter a sleep state when a face image cannot be acquired within a preset time. 68. The method of claim 67,
A door unlocking device for a vehicle, further comprising a second control module for controlling the face recognition module to enter a sleep state when face recognition is not successful within a preset time. 70. The method according to any one of claims 63 to 69,
The unlocking module is
determining a door to which unlocking authority is granted to the target object according to the success of face recognition;
and sending a door unlocking command and/or a door opening command to one or more doors of the vehicle according to a door for which unlocking authority is granted to the target object. 71. The method according to any one of claims 63 to 70,
The face recognition includes biometric detection and face authentication,
The recognition module,
a face authentication module for acquiring the first image by an image sensor of the image acquisition module, and performing face authentication based on the first image and pre-registered facial features;
a vehicle having a biometric detection module configured to acquire a first depth map corresponding to the first image by a depth sensor of the image acquisition module, and perform biometric detection based on the first image and the first depth map; door unlocking device. 72. The method of claim 71,
The biometric detection module,
an update submodule for updating the first depth map based on the first image to obtain a second depth map;
and a determining sub-module configured to determine a biometric detection result of the target object based on the first image and the second depth map. 73. The method of claim 71 or 72,
The image sensor includes an RGB image sensor or an infrared sensor,
The depth sensor comprises a binocular infrared sensor or a time-of-flight TOF sensor. 74. The method of claim 73,
and the TOF sensor uses a TOF module based on an infrared wavelength region. 75. The method according to any one of claims 72 to 74,
The update sub-module,
and updating a depth value of an effective depth pixel in the first depth map based on the first image to obtain the second depth map. 76. The method of any one of claims 72 to 75,
The update sub-module,
determining, based on the first image, depth estimation values of a plurality of pixels in the first image and related information indicating a degree of relevance between the plurality of pixels;
and updating the first depth map based on the depth estimation value of the plurality of pixels and related information to obtain a second depth map. 77. The method of claim 76,
The update sub-module,
determining an effective depth pixel in the first depth map;
obtaining, from among the depth estimation values of the plurality of pixels, a depth estimation value of the depth effective pixel and depth estimation values of a plurality of neighboring pixels of the depth effective pixel;
acquiring a degree of relevance between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel from the related information of the plurality of pixels;
Based on the estimated depth value of the depth effective pixel, the estimated depth values of a plurality of neighboring pixels of the depth effective pixel, and the degree of relevance between the depth effective pixel and the neighboring pixels of the depth effective pixel, the updated depth value of the depth effective pixel is determined Used to determine the vehicle's door unlocking device. 78. The method of claim 77,
The update sub-module,
determining a depth-related value of the depth effective pixel based on the estimated depth value of a neighboring pixel of the depth effective pixel and a degree of relevance between the depth effective pixel and a plurality of neighboring pixels of the depth effective pixel;
and determining a depth value after updating of the depth effective pixel based on the depth estimation value of the depth abolition pixel and the depth-related value. 79. The method of claim 78,
The update sub-module,
Using the degree of relation between the depth effective pixel and each neighboring pixel as the weight of each neighboring pixel, calculating the weighted sum of the estimated depth values of a plurality of neighboring pixels of the depth effective pixel to obtain a depth related value of the depth effective pixel Used, the door unlocking device of the vehicle. 80. The method according to any one of claims 76 to 79,
The update sub-module,
and determining depth estimates of a plurality of pixels in the first image based on the first image and the first depth map. 81. The method of claim 80,
The update sub-module,
and inputting the first image and the first depth map into a depth estimation neural network for processing, and is used to obtain depth estimation values of a plurality of pixels in the first image. 82. The method of claim 80 or 81,
The update sub-module,
performing fusion processing on the first image and the first depth map to obtain a fusion result;
and determining a depth estimate of a plurality of pixels in the first image based on the fusion result. 83. The method of any one of claims 76 to 82,
The update sub-module,
A door unlocking device for a vehicle, used for inputting the first image into a relevance detection neural network and processing it, and obtaining related information of a plurality of pixels in the first image. 84. The method according to any one of claims 72 to 83,
The update sub-module,
acquiring an image of the target object from the first image;
and used to update the first depth map based on the image of the target object. 85. The method of claim 84,
The update sub-module,
acquiring information on key points of the target object in the first image;
a door unlocking device for a vehicle, used for acquiring an image of the target object from the first image based on the keypoint information of the target object. 86. The method of claim 85,
The update sub-module,
performing target detection on the first image to obtain a material area of the target object;
a door unlocking device for a vehicle, used for performing keypoint detection on an image of a material region of the target object to obtain information of a keypoint of the target object in the first image. 87. The method of any one of claims 72 to 86,
The update sub-module,
acquiring a depth map of the target object from the first depth map;
and updating a depth map of the target object based on the first image to obtain the second depth map. 88. The method according to any one of claims 72 to 87,
The determining sub-module,
A door unlocking device for a vehicle, which is used to input and process the first image and the second depth map into a biometric detection neural network, and to obtain a biometric detection result of the target object. 89. The method according to any one of claims 72 to 88,
The determining sub-module,
performing feature extraction processing on the first image to obtain first feature information;
performing feature extraction processing on the second depth map to obtain second feature information;
and used to determine a biometric detection result of the target object based on the first characteristic information and the second characteristic information. 91. The method of claim 89,
The determining sub-module,
performing fusion processing on the first characteristic information and the second characteristic information to obtain third characteristic information;
and used to determine a biometric detection result of the target object based on the third characteristic information. 91. The method of claim 90,
The determining sub-module,
obtaining a probability that the target object is a living body based on the third characteristic information;
and determining a biometric detection result of the target object based on a probability that the target object is a living body. 92. The method according to any one of claims 63 to 91,
The device for unlocking a door of a vehicle, further comprising an activation activation module for activating a password unlocking module mounted on the vehicle to start a flow of unlocking by a password according to a face recognition failure. 93. The method according to any one of claims 63 to 92,
performing registration of the vehicle owner based on the facial image of the vehicle owner acquired by the image acquisition module;
performing remote registration based on the face image of the vehicle owner acquired by the terminal device of the vehicle owner, and transmitting registration information including the face image of the vehicle owner to the vehicle. A door unlocking device for a vehicle, further comprising a registration module. a memory, a face recognition module, an image acquisition module, and a Bluetooth module, wherein the face recognition module is respectively connected to the memory, the image acquisition module and the Bluetooth module, the Bluetooth module comprising: a Bluetooth device of a predetermined identifier; A microprocessor for waking up the face recognition module when Bluetooth pairing/connection is successful or when a Bluetooth device of the predetermined identifier can be searched, and a Bluetooth sensor connected to the microprocessor, A communication interface for accessing the door domain controller is additionally installed in the recognition module, and when face recognition is successful, the in-vehicle face authentication unlocking system transmits control information for unlocking the door to the door domain controller through the communication interface. . 95. The method of claim 94,
wherein the image acquisition module includes an image sensor and a depth sensor. 96. The method of claim 95,
The depth sensor includes a binocular infrared sensor in which two infrared cameras are installed on both sides of a camera of the image sensor. 97. The method of claim 96,
The image acquisition module is a complementary light installed between the infrared camera of the binocular infrared sensor and the camera of the image sensor, and includes at least one of the light beam for the image sensor and the light beam for the depth sensor. In addition, in-vehicle face authentication unlock system. 96. The method of claim 95,
The image acquisition module further includes a laser installed between the camera of the depth sensor and the camera of the image sensor. 99. The method according to any one of claims 94 to 98,
The in-vehicle face authentication unlocking system, which is connected to the face recognition module and further includes a password unlocking module for unlocking the door. 101. The method of claim 99,
The in-vehicle face authentication unlocking system, wherein the password unlocking module includes one or two of a touch screen and a keyboard. 101. The method according to any one of claims 94 to 100,
The in-vehicle face authentication unlocking system further comprising battery modules respectively connected to the microprocessor and the face recognition module. 102. A vehicle, comprising the in-vehicle face authentication unlock system according to any one of claims 94 to 101, wherein the vehicle-mounted face authentication unlock system is connected to a door domain controller of the vehicle. 103. The method of claim 102,
and the image acquisition module is mounted outside a vehicle cabin of the vehicle. 104. The method of claim 103,
and the image acquisition module is mounted on one or more of a B-pillar, one or more doors, and one or more rearview mirrors of the vehicle. 105. The method according to any one of claims 102 to 104,
and the face recognition module is mounted in the vehicle and is connected to the door domain controller via a CAN bus. processor and
a memory for storing instructions executable by the processor;
63. The electronic device, wherein the processor is configured to execute the method of any one of claims 1 to 62. 63. A computer readable storage medium having stored thereon computer program instructions, wherein when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 62 is realized. 63. A computer program comprising computer readable code, wherein the computer readable code, when operated in an electronic device, causes a processor of the electronic device to execute instructions for realizing the method of any one of claims 1 to 62. computer program.

Images