WO2020088029A1

WO2020088029A1 - Liveness detection method, storage medium, and electronic device

Info

Publication number: WO2020088029A1
Application number: PCT/CN2019/100261
Authority: WO
Inventors: 唐宇晨; 邱迪
Original assignee: 北京三快在线科技有限公司
Priority date: 2018-10-29
Filing date: 2019-08-12
Publication date: 2020-05-07
Also published as: CN111104833A

Abstract

The present disclosure is intended to provide a liveness detection method, a storage medium, and an electronic device, so as to solve the problem in related art of low liveness detection accuracy during face recognition. The method comprises: obtaining a face depth map and a face thermal image of an object to be detected; inputting the face depth map and the face thermal image into a feature extraction model, and extracting feature information from the face depth map and the face thermal image by means of the feature extraction model; and determining, according to the feature information and a classification model, whether the object to be detected belongs to a liveness category.

Description

Biopsy method, storage medium and electronic equipment

This application requires the priority of the Chinese patent application filed on October 29, 2018 with the application number 201811269906.8 and the invention titled "Methods and devices for biopsy, storage media and electronic equipment". In this application.

Technical field

The present disclosure relates to the field of image processing, and in particular, to a biopsy method, storage medium, and electronic equipment.

Background technique

With the development of technology, the efficiency of data processing has also increased. Among them, the authentication method of legal identity has also undergone rapid changes. In the related art, a scheme for validating the user's legal identity by collecting the biometrics of the person to be verified is proposed. These biometrics may be fingerprint characteristics or human facial characteristics, and so on.

The human face is innate with other biological characteristics of the human body (fingerprints, irises, etc.), and its uniqueness and good characteristics that are not easily copied provide the necessary premise for identity authentication. Compared with other types of biometrics, face recognition has the characteristics of non-contact, that is to say, the user can obtain face images without directly contacting the device. In addition, in actual application scenarios, multiple faces can be sorted, judged, and recognized, but at present, the accuracy of biopsy is low in face recognition.

Summary of the invention

The purpose of the present disclosure is to provide a biopsy method, storage medium, and electronic equipment to solve the problem of low biopsy accuracy in face recognition in the related art.

To achieve the above objective, in a first aspect, the present disclosure provides a biopsy method, the method including:

Obtain the facial depth map and facial thermal image of the subject to be inspected;

Input the facial depth map and the facial thermal image into a feature extraction model, and extract feature information from the facial depth map and the facial thermal image through the feature extraction model;

According to the feature information and the classification model, it is determined whether the object to be examined belongs to a living body category.

Optionally, the obtaining of the facial depth map and facial thermal image of the object to be inspected includes:

At the same acquisition time, obtain the RGB image, depth map and thermal image of the object to be inspected;

Acquiring facial position coordinates from the RGB image;

In the depth map, an area corresponding to the face position coordinates is used as the face depth map;

In the thermal image, an area corresponding to the facial position coordinates is used as the facial thermal image.

Optionally, the method further includes:

Through the histogram equalization, the contrast of one or more images in the RGB image, the depth image or the thermal image image is enhanced, and the contrast is used to represent the contrast between the face area and the background area.

Optionally, the feature information includes a feature matrix, and the feature extraction model includes a convolutional neural network;

The extracting feature information from the facial depth map and the facial thermal image through the feature extraction model includes:

Fusing pixel points corresponding to positions in the facial depth map and the facial thermal image map to obtain a fused facial image;

Feature extraction is performed on the fused facial image through the convolutional neural network to obtain the feature matrix.

Optionally, the determining, according to the feature information and the classification model, whether the object to be examined belongs to a living body category includes:

Input the feature matrix into the classification model to obtain the probability value that the fused facial image belongs to the living body category;

If the probability value is greater than the probability threshold, it is determined that the object to be examined belongs to the living body category.

Optionally, the classification model includes a fully connected layer and a Softmax classification function;

The inputting the feature matrix into the classification model to obtain the probability value that the fused facial image belongs to the living body category includes:

Input the fully connected layer to the feature matrix and output a multi-dimensional feature vector, wherein the number of dimensions of the multi-dimensional feature vector corresponds to the number of categories of the Softmax classification function;

The probability value that the fused facial image belongs to the living body category is obtained according to the following formula:

Where a _i represents the probability value of the i-th category of the Softmax classification function, and z _i is the i-th value in the multi-dimensional feature vector.

Optionally, the fusing the pixel points corresponding to the positions of the facial depth map and the facial thermal image map to obtain the fused facial image includes:

The value obtained by weighting and averaging the value of the first pixel in the facial depth map and the value of the second pixel in the target image channel in the facial thermal image is used as the fused facial image The value of the pixel corresponding to the position of the first pixel in, wherein the second pixel corresponds to the position of the first pixel, and the target image channel is any image in the facial thermal image One image channel.

In a second aspect, the present disclosure provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the steps performed by the following biopsy method:

Optionally, the program is used to implement the following steps when executed by the processor:

Acquiring facial position coordinates from the RGB image;

Through histogram equalization, the contrast of one or more images in the RGB image, the depth image or the thermal image image is enhanced, and the contrast is used to represent the contrast between the face area and the background area.

When the program is executed by the processor, the following steps are implemented:

In a third aspect, the present disclosure provides an electronic device, including:

Memory, on which computer programs are stored;

The processor is configured to execute the computer program in the memory to implement the steps performed by the following biopsy method:

Optionally, the processor is used to perform the following steps:

Acquiring facial position coordinates from the RGB image;

Optionally, the processor is used to perform the following steps:

The processor is used to perform the following steps:

Optionally, the processor is used to perform the following steps:

The processor is used to perform the following steps:

Optionally, the processor is used to perform the following steps:

BRIEF DESCRIPTION

The drawings are used to provide a further understanding of the present disclosure, and constitute a part of the specification, together with the following specific embodiments to explain the present disclosure, but do not constitute a limitation of the present disclosure. In the drawings:

Fig. 1 is a flow chart of a method for biopsy according to an exemplary embodiment.

Fig. 2 is a flow chart of another method for biopsy according to an exemplary embodiment.

FIG. 3 is a schematic diagram of a convolutional layer neural network shown in FIG. 2.

4 is a schematic diagram of a convolutional layer neural network operation shown in FIG. 2.

Fig. 5 is a block diagram of a device for biopsy according to an exemplary embodiment.

Fig. 6 is a block diagram of an electronic device according to an exemplary embodiment.

detailed description

The specific embodiments of the present disclosure will be described in detail below with reference to the drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure, and are not intended to limit the present disclosure.

In the related art, the face image of the object to be inspected can be collected by taking a picture of the object to be inspected, and the legal identity authentication of the object to be inspected can be performed by the face image of the object to be inspected. Since the authentication technology in the related art is based on 2D (two-dimensional) images, it is impossible to accurately determine whether the object to be examined is a living body. For this authentication technology, some criminals can simulate facial features of legal objects through tools such as masks or three-dimensional face models, thus disguising as legal objects for authentication. In this case, the machine will show that the authentication is passed, allowing the criminals to steal The account information of the legal object is taken, therefore, the authentication technology in the related technology has low accuracy and poor security.

In this regard, the embodiments of the present disclosure provide a biopsy method to improve the accuracy of biopsy during face recognition.

Fig. 1 is a flow chart of a method for biopsy according to an exemplary embodiment. The method can be applied to devices with facial recognition functions, such as mobile terminals, cash dispensers, and other electronic devices. The method includes:

S11. The electronic device obtains a facial depth map and a facial thermal image of the object to be inspected.

Among them, the depth map (DepthMap) is an image or image channel containing information about the distance of the surface of the object to be inspected at the viewpoint. Each pixel value of the depth map is the actual distance of the sensor from the object.

The thermal image is also called infrared thermal image, which can reflect the temperature distribution on the surface of the object.

Specifically, the acquiring of the facial depth map and the thermal image of the subject to be examined includes: simultaneously acquiring the RGB map, the depth map and the thermal image of the subject to be examined; acquiring the facial position coordinates from the RGB image; The areas where the facial position coordinates are mapped to the depth map and the thermal image map are respectively a facial depth map and a facial thermal image map.

In the above process, acquiring the RGB image, depth map and thermal image of the object to be inspected at the same time means that the electronic device acquires the image of the object to be inspected at the same acquisition time to obtain the RGB image, depth map and heat Like picture.

In the above process, mapping the region of the facial position coordinates to the depth map and the thermal image map as the facial depth map and the facial thermal image map respectively means that the electronic device The area corresponding to the facial position coordinates is used as the facial depth map, and in the thermal image, the area corresponding to the facial position coordinates is used as the facial thermal image.

In a specific implementation, the electronic device can simultaneously obtain the RGB image, the depth image, and the thermal image of the object to be inspected through the 3D camera. The RGB image has better facial recognizability, so the facial position coordinates of the object to be inspected can be obtained through the RGB image.

In yet another possible implementation manner, the RGB image, the depth image and the thermal image of the object to be inspected can also be obtained from the video. To ensure the effect of the alignment of the pixel coordinates in the later period, the acquired RGB image, depth image and thermal image The image can correspond to the same frame in the video file.

In addition, since the above three images are acquired at the same time, the pixels of the images are in a corresponding relationship, which is beneficial to the alignment of the pixel coordinates of the three images.

In specific implementation, the above three images can be pre-processed by histogram equalization to enhance the contrast between the face area and the background area, thereby highlighting the effective information corresponding to the facial features. In this way, the accuracy of subsequent feature extraction can be improved.

In the above process, the preprocessing process may include the following steps: the electronic device enhances the contrast of one or more images in the RGB map, the depth map, or the thermal image map through histogram equalization, and the contrast Used to represent the contrast between the face area and the background area.

S12. The electronic device inputs the facial depth map and the facial thermal image into the special effect extraction model, and extracts feature information from the facial depth map and the facial thermal image through the feature extraction model.

For example, corresponding to the nose-eye area of a person, facial features such as the highest point of the face, the lowest point of the face, and the relative position relationship between the highest point and the lowest point can be obtained from the depth map. As another example, the temperature distribution on the face is also different. For example, in winter, the temperature near the mouth and nose is high, and the temperature near the edge of the face is low.

For a living body, the above facial features can form certain laws, and preset living body facial feature information can be generated according to these laws.

For another example, the feature information may include a feature matrix, and the feature extraction model may include a convolutional neural network. In this case, the feature information extraction in S12 may include the following steps: the electronic device fuse the facial depth map Pixels corresponding to the positions in the facial thermal image diagram are used to obtain a fused facial image, and feature extraction is performed on the fused facial image through the convolutional neural network to obtain the feature matrix.

In the above process, it is equivalent to the electronic device extracting the feature matrix of the facial depth map and the facial thermal image through a convolutional neural network. These feature matrices can further input the decision model of the corresponding category corresponding to the feature, that is, the preset classification function model used in conjunction with the pre-trained convolutional neural network. Therefore, the above decision model can also be referred to as "Model", by using a classification model in conjunction with a convolutional neural network, it is possible to determine whether the object to be examined belongs to a living body category.

S13. The electronic device determines whether the object to be inspected belongs to the living body category according to the feature information and the judgment model of the category corresponding to the feature.

In S13, it is equivalent to the electronic device determining whether the object to be examined belongs to the living body category based on the feature information and the classification model. Further, when the electronic device determines whether the object to be examined is a living body category, it may perform the following steps: input the feature matrix into the classification model to obtain a probability value that the fused facial image belongs to the living body category; If the probability value is greater than the probability threshold, it is determined that the object to be examined belongs to the living body category.

Specifically, the judgment model (that is, the classification model) of the corresponding category corresponding to the feature includes preset living facial feature information, and the feature information can be compared with the preset living facial feature information, if the extraction is obtained this time The degree of matching between the feature information and the preset living body facial feature information is greater than a preset threshold, it is determined that the object to be examined is a living body; otherwise, it is determined that the object to be examined is not a living body.

The object to be inspected is non-living means that the object to be inspected is illegal this time. Further, when the operation related to facial recognition is specifically performed, the object to be inspected may be prohibited from accessing the account or generating an alarm message.

The above technical solutions can at least achieve the following technical effects:

By obtaining a facial depth map and a facial thermal image of the object to be inspected, and extracting feature information from the facial depth map and the facial thermal image through a preset feature extraction model, and further according to the feature information, and The judgment model of the corresponding category corresponding to the feature determines whether the object to be inspected belongs to the living body category. In this way, the accuracy of the biopsy is improved, and the mask or facial model can be identified as a legitimate person to invade.

In addition, the above technical solution improves the stability and robustness of the biopsy compared with the biopsy performed solely by thermal imaging.

Compared with the scheme of performing facial recognition by shooting a video instructing the subject to perform a series of motion instructions, the above technical solution does not require special cooperation of the subject to be examined, which improves the user experience and improves the biopsy operation during facial recognition effectiveness.

Fig. 2 is a flow chart of another method for biopsy according to an exemplary embodiment. The method can be applied to devices with facial recognition functions, such as mobile terminals, cash dispensers, and other electronic devices. The method includes:

S21. The electronic device obtains a facial depth map and a facial thermal image of the object to be inspected.

In specific implementation, it is possible to obtain the RGB image, depth map and thermal image of the object to be inspected at the same time through the 3D camera. The RGB image has better facial recognizability, so the facial position coordinates of the object to be inspected can be obtained through the RGB image.

In specific implementation, the above three graphics of the image can be pre-processed by histogram equalization to enhance the contrast between the face area and the background area, thereby highlighting the effective information corresponding to the facial features. In this way, the accuracy of subsequent feature extraction can be improved.

S22. The electronic device fuses the pixel points corresponding to positions in the facial depth map and the facial thermal image map to obtain a fused facial image.

Specifically, the fusing pixel points corresponding to the face depth map and the face thermal image map to obtain a fused face image includes: combining values of first pixels in the face depth map, and the The value obtained by weighted averaging the value of the second pixel in the target image channel in the facial thermal image is used as the value of the pixel corresponding to the position of the first pixel in the fused facial image, where, The second pixel corresponds to the position of the first pixel, and the target image channel is any image channel in the facial thermal image.

For example, if A (i, j) is a pixel in the facial depth map, and B (i, j) is a pixel in a single channel of the facial thermal image, then the fusion pixel C ( i, j) can be obtained by the following formula:

C (i, j) = w _A (i, j) A (i, j) + w _B (i, j) B (i, j)

w _A (i, j) + w _B (i, j) = 1

Weight by adjusting the corresponding portion of the depth map weight w _A (i, j) and the corresponding facial thermography right of FIG weight fused images w _B (i, j), can be more flexibly adjust the resulting depth distribution of elements and temperature distribution factors share The proportion of the face feature matrix makes the extracted facial feature matrix more clearly reflect the facial features of the object to be tested.

The fused facial image can extract the favorable information of the depth map and the thermal image map to the maximum extent, increase the information content of the resulting image, and facilitate more accurate, reliable and comprehensive acquisition of image features.

S23. The electronic device extracts the feature matrix of the fused facial image through a pre-trained convolutional neural network model.

Convolutional neural network is a multi-layer neural network, including convolutional layer and pooling layer. Artificial neurons can respond to the surrounding units, continuously reduce the dimensionality of image recognition problems with huge amounts of data, and finally enable them to be trained to achieve classification, positioning, detection and other functions and functions. In addition, convolutional neural networks can be trained. The training method of convolutional neural network needs to use the chain derivation rule to differentiate the nodes of the hidden layer, that is, the gradient propagation and chain derivation back propagation rules.

The schematic diagram of a convolutional neural network shown in FIG. 3 is a schematic diagram of a structure of a neural network provided by an embodiment of the present disclosure, that is, from an input layer (hidden layer) to an output layer (hidden layer). Among them, the hidden layer includes many different layers (convolutional layer, pooling layer, activation function layer, fully connected layer, etc.).

The feature extraction is completed by downsampling of the convolutional layer, and a feature matrix is generated. Please refer to the operation schematic diagram provided by the embodiment of the present disclosure shown in FIG. 4.

First, the above-mentioned fused facial image may be an input matrix of 7 * 7 in the figure, and each pixel in the matrix is a source pixel. A 3 * 3 filter window traverses the input matrix and performs a convolution operation to obtain the value output after the convolution operation. Among them, the filtering window is also called convolution matrix (convolution kernel).

The specific convolution operation process can be seen in the vertical formula in the upper right corner of the figure, and the operation result is -8.

The center value (pixels with a value of 1) in the window matrix selected in the input matrix in Figure 4 is replaced by the result of the convolution operation, that is, the pixels with a value of -8.

It is worth noting that the above calculation process may be iterated repeatedly as many times as necessary until the required feature matrix is generated.

S24. The electronic device inputs the feature matrix into a preset classification function model (that is, a classification model) to obtain a probability value that the fused facial image belongs to a living body category.

In an optional embodiment, the preset classification function model (that is, the classification model) may include a fully connected layer and a Softmax classification function. Specifically, by inputting the feature matrix into the Softmax classification function, the output of the obtained multiple neurons can be mapped into the interval of 0 to 1. That is to say, the fused image is extracted through the convolutional neural network and input to the Softmax layer, and finally the probability value corresponding to each category is output.

Specifically, before inputting the feature matrix into the Softmax classification function, the image feature matrix can also be fully connected layer transformed, that is to say, the electronic device inputs the feature matrix into the fully connected layer and outputs a multi-dimensional feature vector. The number of dimensions of the multi-dimensional feature vector corresponds to the number of categories of the Softmax classification function.

After the convolution operation in the above example, the output of the neuron can be expressed by the following formula:

Where x _ij is the j-th input value of the i-th neuron; w _ij is the j-th weight of the i-th neuron, b is the offset value, and z _i represents the i-th output of the network, and That is, the i-th value in the multi-dimensional feature vector.

Further, the electronic device may determine the probability value that the fused facial image belongs to the living body category according to the following formula:

Where a _i represents the probability value of the i-th category of Softmax, z _i is the i-th value in the multi-dimensional feature vector, and e is a constant.

For example, the Softmax classification function can have two categories, the first category is "non-living" and the corresponding probability value is a ₁ ; the second category is "living" and the corresponding probability value is a ₂ . Then, through the above probability calculation formula, the probability value a _{2 that} the fused facial image belongs to the living body category can be obtained.

S25. If the probability value is greater than a preset probability threshold, the electronic device determines that the object to be detected belongs to a living body category.

For example, the preset probability threshold is 0.5. If the probability value P belonging to the living body category is greater than 0.5, it can be determined that the object to be tested is a living body; otherwise, the probability value P belonging to the living body category is not greater than 0.5, it is determined that the object to be tested is a non-living body.

It is worth noting that the above method embodiments are described as a series of action combinations for the sake of simple description, but those skilled in the art should know that the present invention is not limited by the described action sequence. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

Fig. 5 is a block diagram of a device for biopsy according to an exemplary embodiment. The device can be applied to devices with facial recognition functions, such as mobile terminals, cash dispensers and other electronic devices. The device includes:

The obtaining module 310 is used to obtain a facial depth map and a thermal image of the subject to be inspected;

A feature extraction module 320, configured to extract feature information from the facial depth map and the facial thermal image through a preset feature extraction model;

The determining module 330 is configured to determine whether the object to be examined belongs to the living body category according to the feature information and the determination model of the corresponding category corresponding to the feature.

Optionally, the acquisition module is used to:

Obtain the RGB image, depth image and thermal image of the object to be inspected at the same time;

Acquiring facial position coordinates from the RGB image;

The regions where the facial position coordinates are mapped to the depth map and the thermal image map are used as the facial depth map and the facial thermal image map, respectively.

Optionally, the acquisition module is used to:

Enhance one of the RGB image, the depth image and the thermal image image by histogram equalization or

Contrast between face area and background area in various images.

Optionally, the feature extraction module is used to fuse pixel points corresponding to the facial depth map and the facial thermal image to obtain a fused facial image; extract the fusion through a pre-trained convolutional neural network model The feature matrix of the facial image;

The determining module is configured to input the image feature matrix into a preset classification function model to obtain a probability value that the fused facial image belongs to a living body class; if the probability value is greater than a preset probability threshold, then It is determined that the object to be examined is a living body.

Optionally, the preset classification function model is a Softmax classification function;

The determination module is used to:

Performing a fully connected layer transformation on the image feature matrix to obtain an output multi-dimensional feature vector, where the number of dimensions of the multi-dimensional feature vector corresponds to the number of categories of the Softmax classification function;

The probability value that the fused facial image belongs to the living body class is determined according to the following formula:

Optionally, the feature extraction module is configured to compare the value of the first pixel in the facial depth map and the second corresponding to the first pixel in the target image channel in the facial thermal image The value obtained by weighted averaging of pixel values is used as the value of the pixel corresponding to the first pixel in the fused facial image, wherein the target image channel is any image in the facial thermal image One image channel.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above-mentioned division of each functional unit (module) is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated by different functional units as needed (Module) is completed, that is, the internal structure of the device is divided into different functional units (modules) to complete all or part of the functions described above. For the specific working processes of the functional units (modules) described above, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the steps performed by the following biopsy method:

Acquiring facial position coordinates from the RGB image;

Where a _i represents the probability value of the i-th category of the Softmax classification function, z _i is the i-th value in the multi-dimensional feature vector, and e is a constant.

An embodiment of the present disclosure provides an electronic device, including:

Memory, on which computer programs are stored;

Optionally, the processor is used to perform the following steps:

Acquiring facial position coordinates from the RGB image;

Optionally, the processor is used to perform the following steps:

The processor is used to perform the following steps:

Optionally, the processor is used to perform the following steps:

The processor is used to perform the following steps:

Optionally, the processor is used to perform the following steps:

Other features and advantages of the present disclosure will be described in detail in the detailed description section that follows.

Fig. 6 is a block diagram of an electronic device according to an exemplary embodiment. As shown in FIG. 6, the electronic device 400 may include: a processor 401 and a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input / output (I / O) interface 404, and a communication component 405.

The processor 401 is used to control the overall operation of the electronic device 400 to complete all or part of the steps in the method for biopsy. The memory 402 is used to store various types of data to support operation on the electronic device 400, and the data may include, for example, instructions for any application or method for operating on the electronic device 400, and application-related data, For example, the pre-trained convolutional neural network model, the thermal image and depth map data of the object to be detected, in addition, it can also include the identity data of legitimate users, messages sent and received, audio, video, and so on. The memory 402 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM for short), electrically erasable programmable read-only memory ( Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), read-only Memory (Read-Only Memory, ROM for short), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 403 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 402 or transmitted through the communication component 405. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 404 provides an interface between the processor 401 and other interface modules. The other interface modules may be a keyboard, a mouse, a button, and so on. These buttons can be virtual buttons or physical buttons. The communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so the corresponding communication component 405 may include: Wi-Fi module, Bluetooth module, NFC module.

In an exemplary embodiment, the electronic device 400 may be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit (ASIC), digital signal processor (DSP), digital signal processing device (Digital Signal Processing (Device DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic components Implementation, for performing the method for biopsy described above.

In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which when executed by a processor implements the steps of the method for biopsy described above. For example, the computer-readable storage medium may be the above-mentioned memory 402 including program instructions, which may be executed by the processor 401 of the electronic device 400 to complete the above-described method for biopsy.

The preferred embodiments of the present disclosure have been described in detail above with reference to the drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure. These simple modifications all fall within the protection scope of the present disclosure. In addition, it should be noted that the specific technical features described in the above specific embodiments can be combined in any suitable manner without contradictions. In order to avoid unnecessary repetition, the present disclosure The combination method will not be explained separately.

In addition, various combinations of various embodiments of the present disclosure can also be arbitrarily combined, as long as it does not violate the idea of the present disclosure, it should also be regarded as what is disclosed in the present disclosure.

Claims

A biopsy method, characterized in that the method includes:

Obtain the facial depth map and facial thermal image of the subject to be inspected;

Input the facial depth map and the facial thermal image into a feature extraction model, and extract feature information from the facial depth map and the facial thermal image through the feature extraction model;

According to the feature information and the classification model, it is determined whether the object to be examined belongs to a living body category.
The method according to claim 1, wherein the acquiring of the facial depth map and the facial thermal image of the object to be inspected comprises:

At the same acquisition time, obtain the RGB image, depth map and thermal image of the object to be inspected;

Acquiring facial position coordinates from the RGB image;

In the depth map, an area corresponding to the face position coordinates is used as the face depth map;

In the thermal image, an area corresponding to the facial position coordinates is used as the facial thermal image.
The method according to claim 2, wherein the method further comprises:

Through histogram equalization, the contrast of one or more images in the RGB image, the depth image or the thermal image image is enhanced, and the contrast is used to represent the contrast between the face area and the background area.
The method according to claim 1, wherein the feature information includes a feature matrix, and the feature extraction model includes a convolutional neural network;

The extracting feature information from the facial depth map and the facial thermal image through the feature extraction model includes:

Fusing pixel points corresponding to positions in the facial depth map and the facial thermal image map to obtain a fused facial image;

Feature extraction is performed on the fused facial image through the convolutional neural network to obtain the feature matrix.
The method according to claim 4, wherein the determining whether the subject to be examined belongs to the living body category according to the feature information and the classification model includes:

Input the feature matrix into the classification model to obtain the probability value that the fused facial image belongs to the living body category;

If the probability value is greater than the probability threshold, it is determined that the object to be examined belongs to the living body category.
The method according to claim 5, wherein the classification model includes a fully connected layer and a Softmax classification function;

The inputting the feature matrix into the classification model to obtain the probability value that the fused facial image belongs to the living body category includes:

Input the fully connected layer to the feature matrix and output a multi-dimensional feature vector, wherein the number of dimensions of the multi-dimensional feature vector corresponds to the number of categories of the Softmax classification function;

The probability value that the fused facial image belongs to the living body category is obtained according to the following formula:

Where a i represents the probability value of the i-th category of the Softmax classification function, and z i is the i-th value in the multi-dimensional feature vector.
The method according to claim 4, wherein the fusing the pixel points corresponding to the positions of the facial depth map and the facial thermal image map to obtain the fused facial image includes:

The value obtained by weighting and averaging the value of the first pixel in the facial depth map and the value of the second pixel in the target image channel in the facial thermal image is used as the fused facial image The value of the pixel corresponding to the position of the first pixel in, wherein the second pixel corresponds to the position of the first pixel, and the target image channel is any image in the facial thermal image One image channel.
A computer-readable storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the following steps performed by the biopsy method are implemented:

Obtain the facial depth map and facial thermal image of the subject to be inspected;

Input the facial depth map and the facial thermal image into a feature extraction model, and extract feature information from the facial depth map and the facial thermal image through the feature extraction model;

According to the feature information and the classification model, it is determined whether the object to be examined belongs to a living body category.
The storage medium according to claim 8, wherein the program is used to implement the following steps when executed by the processor:

At the same acquisition time, obtain the RGB image, depth map and thermal image of the object to be inspected;

Acquiring facial position coordinates from the RGB image;

In the depth map, an area corresponding to the face position coordinates is used as the face depth map;

In the thermal image, an area corresponding to the facial position coordinates is used as the facial thermal image.
The storage medium according to claim 9, wherein the program is used to implement the following steps when executed by the processor:

Through histogram equalization, the contrast of one or more images in the RGB image, the depth image or the thermal image image is enhanced, and the contrast is used to represent the contrast between the face area and the background area.
The storage medium according to claim 8, wherein the feature information includes a feature matrix, and the feature extraction model includes a convolutional neural network;

When the program is executed by the processor, the following steps are implemented:

Fusing pixel points corresponding to positions in the facial depth map and the facial thermal image map to obtain a fused facial image;

Feature extraction is performed on the fused facial image through the convolutional neural network to obtain the feature matrix.
The storage medium according to claim 11, wherein the program is used to implement the following steps when executed by the processor:

Input the feature matrix into the classification model to obtain the probability value that the fused facial image belongs to the living body category;

If the probability value is greater than the probability threshold, it is determined that the object to be examined belongs to the living body category.
The storage medium according to claim 12, wherein the classification model includes a fully connected layer and a Softmax classification function;

When the program is executed by the processor, the following steps are implemented:

Input the fully connected layer to the feature matrix and output a multi-dimensional feature vector, wherein the number of dimensions of the multi-dimensional feature vector corresponds to the number of categories of the Softmax classification function;

The probability value that the fused facial image belongs to the living body category is obtained according to the following formula:

Where a i represents the probability value of the i-th category of the Softmax classification function, and z i is the i-th value in the multi-dimensional feature vector.
An electronic device, characterized in that it includes:

Memory, on which computer programs are stored;

The processor is configured to execute the computer program in the memory to implement the steps performed by the following biopsy method:

Obtain the facial depth map and facial thermal image of the subject to be inspected;

Input the facial depth map and the facial thermal image into a feature extraction model, and extract feature information from the facial depth map and the facial thermal image through the feature extraction model;

According to the feature information and the classification model, it is determined whether the object to be examined belongs to a living body category.
The electronic device according to claim 14, wherein the processor is configured to perform the following steps:

At the same acquisition time, obtain the RGB image, depth map and thermal image of the object to be inspected;

Acquiring facial position coordinates from the RGB image;

In the depth map, an area corresponding to the face position coordinates is used as the face depth map;

In the thermal image, an area corresponding to the facial position coordinates is used as the facial thermal image.
The electronic device according to claim 15, wherein the processor is configured to perform the following steps:

Through histogram equalization, the contrast of one or more images in the RGB image, the depth image or the thermal image image is enhanced, and the contrast is used to represent the contrast between the face area and the background area.
The electronic device according to claim 14, wherein the feature information includes a feature matrix, and the feature extraction model includes a convolutional neural network;

The processor is used to perform the following steps:

Fusing pixel points corresponding to positions in the facial depth map and the facial thermal image map to obtain a fused facial image;

Feature extraction is performed on the fused facial image through the convolutional neural network to obtain the feature matrix.
The electronic device according to claim 17, wherein the processor is configured to perform the following steps:

Input the feature matrix into the classification model to obtain the probability value that the fused facial image belongs to the living body category;

If the probability value is greater than the probability threshold, it is determined that the object to be examined belongs to the living body category.
The electronic device according to claim 18, wherein the classification model includes a fully connected layer and a Softmax classification function;

The processor is used to perform the following steps:

Input the fully connected layer to the feature matrix and output a multi-dimensional feature vector, wherein the number of dimensions of the multi-dimensional feature vector corresponds to the number of categories of the Softmax classification function;

The probability value that the fused facial image belongs to the living body category is obtained according to the following formula:

Where a i represents the probability value of the i-th category of the Softmax classification function, and z i is the i-th value in the multi-dimensional feature vector.
The electronic device according to claim 17, wherein the processor is configured to perform the following steps:

The value obtained by weighting and averaging the value of the first pixel in the facial depth map and the value of the second pixel in the target image channel in the facial thermal image is used as the fused facial image The value of the pixel corresponding to the position of the first pixel in, wherein the second pixel corresponds to the position of the first pixel, and the target image channel is any image in the facial thermal image One image channel.