CN110544233A - Depth Image Quality Evaluation Method Based on Face Recognition Application - Google Patents

Abstract

The present invention provides a method for evaluating the quality of depth images based on face recognition applications. The method includes: acquiring depth images of one or more test molds based on a depth perception sensor; wherein, each of the test molds has a different surface shape; for any One described test mould, analyze the depth image of this test mould, extract the feature of described depth image; Wherein, the feature extracted from the depth image of each described test mould is different; According to the depth of all described test moulds The feature of the image is to evaluate the depth image of the face collected by the depth perception sensor. The features extracted by the present invention are more comprehensive and accurate, and the evaluation of the human face depth image based on the extracted features is more accurate, and has high practicability and reproducibility.

Description

Depth Image Quality Evaluation Method Based on Face Recognition Application

technical field

The invention belongs to the technical field of depth perception imaging, and in particular relates to a depth image quality evaluation method based on face recognition applications.

Background technique

Depth-sensing imaging technology has always been an important topic in the subject of machine vision. The current mainstream depth-sensing imaging technologies include single-purpose spatially encoded structured light depth imaging technology, binocular structured light texture-enhanced depth imaging technology and TOF (Time of flight, flight time) technology, etc.

In the field of face recognition applications, with the development of deep learning technology in recent years, it has gradually transitioned from the single use of color image or infrared image data to the comprehensive use of color image and depth image data, or the comprehensive use of infrared image and depth image data. face recognition. Among them, the acquisition of the depth image is mainly through the aforementioned depth-sensing imaging technology, so the quality of the depth image generated based on the depth imaging algorithm has a very critical impact on the application of face recognition.

At present, the relevant algorithms of depth-sensing imaging technology are quite mature. The currently more commonly used depth imaging error measurement method is mainly to measure the registration error between the actual point cloud of the object obtained by the depth imaging algorithm and the real point cloud of the object. This method has the following defects And problems: (1) New errors will be introduced in the process of point cloud registration to affect the evaluation of the quality of depth imaging algorithms; (2) Registration errors will mix various errors together, which cannot be more detailed evaluation according to the needs of practical applications; ( 3) The point cloud registration error only reflects the ability of the depth imaging algorithm to obtain depth data similar to the real object. For many practical applications, the degree of realism is not the main requirement.

To sum up, there is currently no uniform and effective quality evaluation method for depth imaging algorithms in the industry, especially for face recognition applications. The depth imaging algorithm and face recognition algorithm jointly affect the accuracy of face recognition, and it is necessary to evaluate the quality of depth imaging algorithms. Reasonable evaluation, decoupling of depth imaging algorithm and face recognition algorithm. Commonly used depth imaging error measurement methods cannot meet the needs of practical applications, especially for some specific applications, such as the quality assessment of depth imaging algorithms for face recognition. For face recognition applications, it is impossible to accurately measure the quality of the depth imaging algorithm and its impact on the accuracy of face recognition simply through common methods such as point cloud registration error measurement.

Contents of the invention

In order to overcome the problem that the existing depth imaging error measurement methods cannot effectively evaluate the quality of depth images in face recognition applications or at least partially solve the above problems, embodiments of the present invention provide a depth image quality measurement method based on face recognition applications. evaluation method.

An embodiment of the present invention provides a method for evaluating the quality of a depth image based on a face recognition application, including:

Obtaining depth images of one or more test molds based on a depth perception sensor; wherein each of the test molds has a different surface shape;

For any of the test moulds, analyzing the depth images of the test moulds, extracting features of the depth images; wherein, the features extracted from the depth images of each of the test moulds are different;

According to the characteristics of the depth images of all the test moulds, the depth images of the human face collected by the depth perception sensor are evaluated.

Embodiments of the present invention provide a method for evaluating the quality of depth images based on face recognition applications. The method uses a depth sensor to obtain depth images of test molds, analyzes depth images of different surface shapes, and extracts the depth images according to the surface shape of the test mold. Extract the corresponding features from the depth image of the test mold, and use the features of all the depth images of the test mold as evaluation indicators to evaluate the face depth image. The extracted features are more comprehensive and accurate, and the face depth image evaluation based on the extracted features More accurate, and highly practical and reproducible.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic diagram of the overall flow of a method for evaluating the quality of a depth image based on a face recognition application provided by an embodiment of the present invention;

2 is a schematic diagram of the horizontal sinusoidal waveform size of the sinusoidal mold in the depth image quality evaluation method based on the face recognition application provided by the embodiment of the present invention;

3 is a schematic diagram of the vertical sinusoidal waveform size of the sinusoidal mold in the depth image quality evaluation method based on the face recognition application provided by the embodiment of the present invention;

4 is a schematic diagram of the size of the folding mold in the depth image quality evaluation method based on the face recognition application provided by the embodiment of the present invention;

5 is a schematic diagram of the size of a cylindrical surface mold in the depth image quality evaluation method based on the face recognition application provided by the embodiment of the present invention;

FIG. 6 is a schematic diagram of an overall structure of an electronic device provided by an embodiment of the present invention.

Detailed ways

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

In one embodiment of the present invention, a method for evaluating the quality of a depth image based on a face recognition application is provided. FIG. 1 is a schematic diagram of the overall flow of a method for evaluating the quality of a depth image based on a face recognition application provided by an embodiment of the present invention. The method includes : S101, acquiring depth images of one or more test molds based on a depth perception sensor; wherein, each of the test molds has a different surface shape;

Among them, the depth perception sensor is a perception system with a built-in depth imaging algorithm, such as a depth camera integrated with a depth imaging algorithm. The surface shape of the test mold is determined according to the application scenario. For example, when it is applied to face recognition, the surface shape of the test mold is set according to the surface characteristics of the human face. Since the surface features of the human face are relatively complex, the surface features of the human face can be represented by multiple simple surface features, and a test mold can be prepared according to each simple surface feature.

Prepare the test mold first, which can be prepared by 3D printing. A depth image of each test die is then acquired using a depth-aware sensor. The relative positions of the depth perception sensor and the test mold are fixed to ensure that the test mold appears in the center of the field of view of the depth perception sensor without rotation, and is kept parallel to the imaging sensor of the depth perception sensor. Preferably, the relative position between the depth sensing sensor and the test mold can be adjusted using a fixed tooling, and the rotation angle should not be greater than 3°. One depth image of each test mold is acquired using a depth-aware sensor. It is also possible to start from the preset shortest evaluation distance, collect a depth image at a certain distance for each test mold, and analyze the depth images collected at each evaluation distance until the preset farthest evaluation distance, so that each The quality of the depth images acquired by the evaluation distance is evaluated.

S102. For any of the test molds, analyze the depth image of the test mold to extract features of the depth image; wherein, the features extracted from the depth images of each of the test molds are different;

Since the surface shape of each test mold is different, the features extracted from the depth images of each test mold are also different. This embodiment is not limited to the types of features extracted from the depth images of each test mold.

S103. Evaluate the depth images of the human face collected by the depth perception sensor according to the features of the depth images of all the test molds.

Since the surface shape of each test mold is used to represent a simple surface feature of the face, the features of the depth image of each test mold are used as the features of the depth image of the face, thereby obtaining various features of the depth image of the face. Various features of the face depth image are used as evaluation indexes to evaluate the face depth image.

In this embodiment, the depth images of the test molds are obtained by first using the depth sensor, the depth images of different surface shapes are analyzed, and the corresponding features are extracted from the depth images of the test molds according to the surface shape of the test molds. The feature of the image is used as an evaluation index to evaluate the depth image of the face. The extracted features are more comprehensive and accurate. The evaluation of the depth image of the face based on the extracted features is more accurate, and has a high degree of practicability and reproducibility.

On the basis of the above embodiments, the test molds in this embodiment include plane molds, sinusoidal molds, folding molds and cylindrical molds; wherein, the plane mold is a mold whose surface fluctuation error is less than the first preset threshold ; The sinusoidal surface mold is a mold with a horizontal and vertical staggered sinusoidal shape on the surface; the folded surface mold is a mold with a continuous right-angle folding surface; the cylindrical surface mold is a mold with a continuous cylindrical curved surface.

Among them, the plane mold is a plane plate or wall whose plane fluctuation error is less than the first preset threshold, which is used to investigate the accuracy of the depth imaging algorithm for plane feature restoration; the surface shape of the sinusoidal mold is staggered sinusoidal, which is used to simulate Some features of the face, such as nose and mouth; the surface type of the folded surface mold is a continuous right-angle folded surface, which is used to investigate the ability of the depth imaging algorithm to distinguish the normal direction of the plane; the surface type of the cylindrical surface mold is a continuous cylindrical surface, which is used to investigate The smoothness of continuous surface feature recovery by the depth imaging algorithm.

The above-mentioned test mold can be made by 3D printing to ensure that the processing accuracy of the test mold is not less than 1mm. Among them, the overall size of the planar mold is 300mm*300mm, and the first preset threshold is 1mm; the overall size of the sinusoidal mold is 300mm*300mm, and there are 8 peaks in the horizontal and vertical directions. Among them, as shown in Figure 2, the amplitude of the horizontal sine wave is 10mm and the period is 40mm; as shown in Figure 3, the amplitude of the vertical sine wave is 10mm and the period is 30mm; the overall size of the folding mold is 300mm* 300mm, composed of 6 groups of folded surfaces with an included angle of 90 degrees, where the peak-to-valley value of the folded surfaces is 20mm, and the distance between adjacent peaks and peaks is 40mm, as shown in Figure 4; the overall size of the cylindrical surface mold is 300mm*300mm, It consists of three groups of continuous cylindrical surfaces, where the peak-to-valley value of the cylindrical surface is 40mm, and the distance between adjacent peaks and peaks is 80mm, as shown in Figure 5.

On the basis of the above-mentioned embodiments, the features corresponding to the planar mold in this embodiment include precision, void ratio in the effective area, and the percentage of dead pixels; the features corresponding to the sinusoidal mold include sine fit, relative amplitude error and periodic relative error; the features corresponding to the folded surface mold include the normal distinction of right-angled folded surfaces; the features corresponding to the cylindrical surface mold include the normal smoothness of the cylindrical surface.

Specifically, the extracted features of the depth image of the planar mold include three items: precision, void ratio in effective area, and proportion of dead pixels. Among them, the precision feature is used to characterize the precision of the depth imaging algorithm for plane feature restoration, the effective area void ratio feature is used to represent the probability of voids when the depth imaging algorithm restores plane features, and the dead point ratio feature is used to characterize the depth imaging algorithm recovery. The proportion of points with larger errors in plane features.

The features extracted from the depth image of the sinusoidal surface mold include three items: the degree of sinusoidal fit, the relative error of the amplitude and the relative error of the period. Among them, the sine fit feature indicates whether the surface shape restoration is true and accurate during depth imaging, the amplitude relative error feature indicates whether the depth imaging algorithm can identify fluctuations and the correctness of the depth data, and the cycle relative error represents the frequency of the depth imaging algorithm Response, whether it can respond to the high-frequency characteristics of the sinusoidal surface mold.

The feature of extracting the depth image of the folded surface mold includes the discrimination of the normal of the right-angled folded surface, which represents the degree of distinction between the two planes forming the right angle recovered by the depth imaging algorithm. The weighted average distance between them is represented, and the smaller the value of the discrimination degree is, the higher the discrimination ability of the right-angle plane restored by the depth imaging algorithm is.

The feature of extracting the depth image of the cylindrical surface mold includes the normal smoothness of the cylindrical surface, which represents the smoothness of the cylindrical surface features restored by the depth imaging algorithm. The measured value of the normal vector of the three-dimensional point in the selected area is located in the normal direction. The smoothness value is represented by the proportion within a certain range of the value curve, and the smaller the smoothness value is, the less smooth the cylindrical surface recovered by the depth imaging algorithm is.

On the basis of the above embodiments, in this embodiment, for any of the test molds, analyzing the depth image of the test mold, extracting the features of the depth image specifically includes: if the test mold is a planar mold, then The depth image of the test mold is sampled at a preset sampling interval; the coordinate values of the pixel points in the neighborhood window of each sampling point are fitted to a spatial plane based on the least squares method, and the fitting plane corresponding to each of the sampling points is obtained; The distance from any of the sampling points to the fitting plane corresponding to the sampling point is used as the residual error of the objective function at the sampling point; the average value of the residual error of the objective function at all the sampling points is used as the corresponding Precision; the residual error of the objective function at each described sampling point is converted into a pixel error; the ratio of the number of sampling points whose pixel error is greater than the second preset threshold in the depth image of the test mold is used as the badness corresponding to the test mold Point rate; From the center of the effective area of the depth image of the test mold, select the area of preset ratio as the area of interest; count the ratio of the number of hollow points in the area of interest to the total number of pixels in the area of interest, and the The ratio is taken as the effective area void rate corresponding to the test mold.

Specifically, the specific process of extracting precision features is: sampling every preset number of pixels on the depth image collected at each evaluation distance, for example, the sampling interval is 10. Use the least square method to fit the spatial plane according to the coordinate values of the pixels in the neighborhood window of each sampling point, for example, the size of the neighborhood window is 50*50. Obtain the residual of the objective function at each sampling point, that is, the spatial distance from each sampling point to the fitting plane. The average of the spatial distances from all sampling points to the fitting plane is used as the precision feature at each sampling distance.

The specific process of extracting the void rate feature of the effective area is as follows: select an area with a preset ratio from the center of the effective area of the depth image at each evaluation distance of the plane mold as the area of interest, and count the number of void points in the area of interest accounting for the area of interest The percentage of the total number of pixels that characterizes the effective area hole rate for each evaluation distance. Preferably, in this embodiment, 80% of the center of the effective region is used as the region of interest.

The specific process of extracting the proportion of dead pixels is as follows: on the depth image of each evaluation distance, select a pixel every preset number of pixels, for example, the sampling interval is 10. Use the least square method to perform plane fitting on the pixels in the neighborhood window of each sampling point, for example, the size of the neighborhood window is 50*50. Obtain the residual error of the objective function at each sampling point, and then convert the above residual error into a pixel error, and convert the residual error of the objective function at each sampling point into a pixel error by the following formula:

E _p =R _e *T*F/d ² ;

Among them, E _p is the pixel error of any sampling point, _Re is the residual error of the objective function at any sampling point, T is the baseline length of the depth perception sensor, F is the focal length of the depth perception sensor, and d is the acquisition of the depth image distance. The percentage of sampling points whose pixel error is greater than the second preset threshold in the depth image of the plane mold is taken as the dead pixel rate corresponding to the plane mold, for example, the second preset threshold is 0.5.

On the basis of the above embodiments, in this embodiment, for any of the test molds, analyzing the depth image of the test mold, extracting the features of the depth image specifically includes: if the test mold is a sinusoidal surface mold, then Convert the depth image of the test mold into a pseudo-color map; select a section where a plurality of horizontal and a plurality of vertical peaks and valleys are located in the pseudo-color map, connect the peaks at both ends of each section, and obtain the peak connection line and The intersection line of the imaging plane of the depth image; the pixel coordinates on the intersection line are converted into plane coordinates; the converted coordinate points are fitted with a straight line, and the converted coordinates are adjusted according to the slope of the fitted straight line The point is rotated; the rotated coordinate points are fitted with a sinusoidal curve, and according to the fitted sinusoidal curve and the parameter value of the test mold, calculate the relative amplitude error and cycle relative error corresponding to the test mold; The fitting value corresponding to each coordinate point on the line on the fitted sinusoidal curve is used to calculate the corresponding sinusoidal fitting degree of the test mold.

Specifically, the step of feature extraction of the depth image of the sinusoidal mold specifically includes:

a. Convert the depth image of the collected sinusoidal mold into a pseudo-color image, so as to accurately select the peak point of the sinusoidal wave, that is, the extreme point. Select a section where several horizontal and vertical peaks and valleys are located, and obtain the intersection line between the line and the imaging plane by selecting the connection line of the peak points at both ends of the section, and select the image point coordinates (i, j , z) into plane coordinate points (d _ij , z) for subsequent straight line and curve fitting, the conversion formula is described as follows:

Among them, i and j correspond to the horizontal and vertical coordinates of the pixel point, z is the depth value of the pixel point, d _ij is the plane coordinate value, c _x and _cy are the principal point coordinates of the depth camera, f _x and f _y are the depth camera The focal length; preferably, this embodiment selects the peak-to-peak connecting lines of the two transverse and two vertical sections in the central area of the sinusoidal mold for testing.

b. Fit the converted coordinate points on each connection line to a straight line first, and rotate in the opposite direction according to the slope of the straight line, so as to eliminate the influence of the tilting on the sinusoidal fitting effect when the sinusoidal mold is placed when shooting;

c. Both the fitting of the sinusoidal curve and the straight line adopt the least square method. The fitting of the sinusoidal curve requires a good initial value of parameters, including: amplitude, period, phase shift and amplitude shift, which can be manually set or default. way;

d. According to the parameter values of the fitted sinusoidal curve, that is, the amplitude and period, combined with the true value of the parameters of the sinusoidal mold, calculate the relative error of the amplitude and the relative error of the period, and at the same time obtain the simulated curve corresponding to each point on the peak-peak line The combined value is used to calculate the sine fit.

Preferably, the sine fit feature can use the goodness of fit R ² , and its calculation formula is as follows:

Among them, R ² is the sinusoidal fitting degree corresponding to the sinusoidal surface mold, y _i is the fitting value of the i-th coordinate point on the peak line, and Y _i is the actual value of the i-th coordinate point on the wave-peak line, is the average value of the actual values of all coordinate points on the peak line.

On the basis of the above embodiments, in this embodiment, for any of the test molds, analyzing the depth image of the test mold, extracting the features of the depth image specifically includes: if the test mold is a folding mold, then Convert the depth image of the test mold into a pseudo-color map; frame the test area from the pseudo-color map, and convert the non-zero pixel points of the depth in the frame-selected test area according to the parameters of the depth perception sensor to Three-dimensional coordinate point; for any of the three-dimensional coordinate points, based on the KD-tree algorithm, the nearest neighbor points of the preset number in the neighborhood of the three-dimensional coordinate point are obtained, and the plane fitting is carried out to the nearest neighbor points, according to the fitted plane Obtain the normal vector of the three-dimensional coordinate point, and normalize the normal vector to obtain the normal measurement value of the three-dimensional coordinate point; on the unit normal sphere, select two phases of the test mold at right angles The normal true value of the adjacent plane; calculate the Euclidean distance between the normal measured value of the three-dimensional coordinate point and the two described normal true values respectively, and obtain the minimum value in the two described Euclidean distances; The minimum values corresponding to the three-dimensional coordinate points are sorted in ascending order, and the weighted average distance between the normal measurement value of the three-dimensional coordinate points and the normal true value is calculated according to the sorting results, and the weighted average The distance is used as the discrimination degree of the normal line of the right-angled folded surface corresponding to the test mold.

Specifically, the step of performing feature extraction on the depth image of the folding mold specifically includes:

a. Convert the collected folding mold depth map into a pseudo-color map, so as to accurately frame the test area; select the non-zero depth pixels in the frame-selected test area, and convert them into three-dimensional points by combining the parameters of the depth perception sensor cloud;

b. Based on the KD-tree algorithm, the preset number of nearest neighbor points in the neighborhood of each 3D point is obtained. By fitting the plane to the above nearest neighbor points, the normal vector of each 3D point is calculated and normalized. After traversing the 3D point cloud , to remove outlier noise points through Euclidean clustering on the obtained unit normal point cloud. The unit normal point of each three-dimensional point obtained after denoising is used as the normal measurement value of each three-dimensional coordinate point. Preferably, in this embodiment, 50 nearest neighbor points in the neighborhood of each three-dimensional point are used for plane fitting;

c. On the unit normal sphere, select the normal true values of two vertical planes, that is, the normal point cloud aggregation points, traverse the Euclidean distance between each normal measurement value and the two normal true values, and take two Euclidean The minimum value in the distance is sorted from small to large, and the corresponding weight is set according to the percentage, and the weighted average distance is calculated, that is, the average value is taken after the weighted distance is summed, and it is used as the distinguishing feature of the rectangular folded surface normal. The formula is as follows:

In the formula, D is the weighted average distance, a is the distance sorting percentage used in weighting, α and β are the weights of the corresponding percentages, d _i is the normal measurement value of the i-th three-dimensional coordinate point in the sorting result and the two method The minimum value in the Euclidean distance between the true values, d _b is the Euclidean distance between the two normal true values; preferably, the present embodiment adopts the distance weight of the first 80% of the sorting to be set to 0.2, and the latter 20% The distance weight of is set to 0.8.

On the basis of the above embodiments, in this embodiment, for any of the test molds, analyzing the depth image of the test mold, extracting the features of the depth image specifically includes: if the test mold is a cylindrical mold, then Convert the depth image of the test mold into a pseudo-color map; frame the test area from the pseudo-color map, and convert the non-zero pixel points of the depth in the frame-selected test area according to the parameters of the depth perception sensor to Three-dimensional coordinate point; for any of the three-dimensional coordinate points, based on the KD-tree algorithm, the nearest neighbor points of the preset number in the neighborhood of the three-dimensional coordinate point are obtained, and the plane fitting is carried out to the nearest neighbor points, according to the fitted plane Obtain the normal vector of the three-dimensional coordinate point, and normalize the normal vector to obtain the normal measurement value of the three-dimensional coordinate point; Projecting on the XY coordinate plane to obtain the projection point of the normal measurement value of the three-dimensional coordinate point; fitting the projection points of the normal measurement value of all the three-dimensional coordinate points into a straight line, and calculating each of the projection points to the straight line Calculate the ratio of the projection points whose distance is less than the third preset threshold to all the projection points, and use the ratio as the normal smoothness of the cylindrical surface corresponding to the test mold.

Specifically, the step of extracting features from the depth image of the cylindrical mold specifically includes:

a. Convert the collected depth map of the cylindrical surface mold into a pseudo-color map, so as to accurately frame the test area, select the non-zero depth pixels in the frame area, and convert it into a 3D point cloud in combination with camera parameters;

b. Obtain a certain number of nearest neighbor points in the neighborhood of each 3D point based on the KD-tree algorithm. By fitting the plane to the above nearest neighbor points, calculate and normalize the normal vector of each 3D point. After traversing the 3D point cloud, The unit normal point cloud of each unit point obtained is removed by Euclidean clustering to remove outlier noise points, and the unit normal point of each three-dimensional point obtained after denoising is used as the normal measurement value of each three-dimensional coordinate point; preferably, In this embodiment, 50 nearest neighbor points in the neighborhood of each three-dimensional point are used for plane fitting;

c. On the unit normal sphere, project the normal measurement value to the XY coordinate plane, that is, only take out the x and y coordinates, and perform straight line fitting on the projection points of all three-dimensional points based on the least squares method, and calculate each projection point to For the distance of the fitted line, the ratio of the projected points smaller than the distance threshold to the total number of points is counted, which is the normal smoothness of the cylindrical surface; preferably, the third preset threshold is 0.1.

For face recognition applications, different features of different test molds have different impacts on the quality of the depth imaging algorithm. According to the actual impact of each feature, it is classified as follows:

(1) The first layer of features: sinusoidal mold -> sinusoidal fitting degree, the larger the value of the feature, the better, and it needs to be kept above a certain threshold to ensure a higher face recognition rate; preferably, the proposed method used in the present invention The degree threshold is 0.9;

(2) The second layer of features: sinusoidal mold -> periodic relative error, the closer the feature value is to the same feature of the depth imaging algorithm that generates the face recognition training data set, the better; folded surface mold -> right angle folded surface normal differentiation , the smaller the characteristic value, the better; the cylindrical surface mold->cylindrical surface normal smoothness, the larger the characteristic value, the better;

(3) The third layer of features: flat mold -> proportion of dead pixels, the smaller the value of the feature, the better; flat mold -> void ratio in the effective area, the smaller the better; plane mold -> precision, the The smaller the characteristic value, the better;

(4) The fourth layer feature: sinusoidal mold -> relative amplitude error, the smaller the value of this feature, the better.

According to the impact relationship of features at all levels on face recognition applications, the respective requirements are as follows:

(1) The first layer of features must be satisfied, otherwise it may lead to a low face recognition rate;

(2) The second layer features have a great influence on the face recognition rate, try to be as close as possible to the target value;

(3) The impact of the third-level features on the face recognition rate is not obvious, but it is still very important. Pay special attention to certain features that will seriously affect the face recognition results after exceeding a certain threshold, such as the hole rate, etc.;

(4) The fourth layer feature has no obvious impact on the face recognition rate at present, but try to keep it within the controllable range to avoid affecting the face recognition rate.

This embodiment provides an electronic device. FIG. 6 is a schematic diagram of the overall structure of the electronic device provided by the embodiment of the present invention. The device includes: at least one processor 601, at least one memory 602, and a bus 603; wherein,

The processor 601 and the memory 602 communicate with each other through the bus 603;

The memory 602 stores program instructions that can be executed by the processor 601, and the processor calls the program instructions to perform the methods provided by the above-mentioned method embodiments, for example, including: acquiring depth images of one or more test molds based on a depth perception sensor; wherein , the surface shape of each of the test molds is different; for any of the test molds, the depth image of the test mold is analyzed, and the features of the depth image are extracted; wherein, from the depth image of each of the test molds The extracted features are different; according to the features of the depth images of all the test molds, the depth images of the face collected by the depth perception sensor are evaluated.

This embodiment provides a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores computer instructions. The computer instructions cause the computer to execute the methods provided in the above-mentioned method embodiments, for example, including: Depth images of one or more test moulds; wherein, each of the test moulds has a different surface shape; for any of the test moulds, the depth images of the test moulds are analyzed to extract features of the depth images; wherein , the features extracted from the depth images of each of the test molds are different; according to the features of all the depth images of the test molds, the face depth images collected by the depth perception sensor are evaluated.

Those of ordinary skill in the art can understand that all or part of the steps for realizing the above-mentioned method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the It includes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative efforts.

Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. A depth image quality evaluation method based on face recognition application is characterized by comprising the following steps:

Acquiring depth images of one or more test molds based on a depth perception sensor; wherein the surface shape of each of the test molds is different;

for any test mould, analyzing the depth image of the test mould, and extracting the characteristics of the depth image; wherein the features extracted from the depth image of each of the test molds are different;

and evaluating the face depth image collected by the depth perception sensor according to the characteristics of the depth images of all the test molds.

2. the depth image quality evaluation method based on face recognition application of claim 1, wherein the test mold comprises a plane mold, a sine surface mold, a folded surface mold and a cylindrical surface mold;

The plane mould is a mould with a surface undulation error smaller than a first preset threshold value;

The sine surface die is a die with the surface in a transversely and longitudinally staggered sine shape;

The surface folding mold is a mold with a continuous right-angle folding surface;

the cylindrical surface mould is a mould with a continuous cylindrical curved surface on the surface.

3. the depth image quality evaluation method based on face recognition application of claim 2, wherein the characteristics corresponding to the planar mold comprise precision, effective area voidage and dead pixel ratio;

the characteristics corresponding to the sine surface mold comprise sine fitting degree, amplitude relative error and period relative error;

The characteristics corresponding to the folded surface mould comprise a right-angle folded surface normal line division degree;

the corresponding features of the cylindrical surface mold include a cylindrical surface normal smoothness.

4. the depth image quality evaluation method based on face recognition application according to claim 2, wherein for any test mold, analyzing the depth image of the test mold, and extracting the features of the depth image specifically comprises:

If the test mold is a plane mold, sampling the depth image of the test mold at a preset sampling interval;

Performing space plane fitting on coordinate values of pixel points in a neighborhood window of each sampling point based on a least square method to obtain a fitting plane corresponding to each sampling point;

Taking the distance from any sampling point to a fitting plane corresponding to the sampling point as the residual error of the target function at the sampling point;

Taking the average value of the residual errors of the target functions at all the sampling points as the precision corresponding to the test mold;

converting the residual error of the objective function at each sampling point into a pixel error;

taking the number proportion of sampling points with pixel errors larger than a second preset threshold value in the depth image of the test mold as the dead pixel rate corresponding to the test mold;

selecting a region with a preset proportion from the center of the effective region of the depth image of the test mold as a region of interest;

and counting the proportion of the number of the holes in the region of interest to the total number of the pixels in the region of interest, and taking the proportion as the effective region hole rate corresponding to the test mold.

5. the method of claim 4, wherein the residual error of the objective function at each sampling point is converted into a pixel error by the following formula:

E＝R*T*F/d；

And the parameter is the pixel error of any sampling point, Re is the residual error of an objective function at any sampling point, T is the length of a base line of the depth perception sensor, F is the focal length of the depth perception sensor, and d is the acquisition distance of the depth image.

6. the depth image quality evaluation method based on face recognition application according to claim 2, wherein for any test mold, analyzing the depth image of the test mold, and extracting the features of the depth image specifically comprises:

If the test mold is a sine-surface mold, converting the depth image of the test mold into a pseudo-color image;

Selecting cross sections where a plurality of transverse and longitudinal peak valleys are located from the pseudo-color image, connecting the peaks at two ends of each cross section, and obtaining an intersection line of a peak connecting line and an imaging plane of the depth image;

Converting the pixel coordinates on the intersection line into plane coordinates;

Performing linear fitting on the converted coordinate points, and rotating the converted coordinate points according to the slope of the fitted linear;

carrying out sine curve fitting on the rotated coordinate points, and calculating amplitude relative errors and period relative errors corresponding to the test mould according to the fitted sine curve and the parameter values of the test mould;

And calculating the corresponding sine fitting degree of the test mold according to the corresponding fitting value of each coordinate point on the peak connecting line on the fitted sine curve.

7. The method of claim 6, wherein the degree of fitting of the sine corresponding to the test mold is calculated according to the fitting value of each coordinate point on the peak connecting line on the fitted sine curve by the following formula:

wherein, R2 is the sine fitting degree corresponding to the test mold, Yi is the fitting value of the ith coordinate point on the peak connecting line, Yi is the actual value of the ith coordinate point on the peak connecting line, and Yi is the average value of the actual values of all coordinate points on the peak connecting line.

8. The depth image quality evaluation method based on face recognition application according to claim 2, wherein for any test mold, analyzing the depth image of the test mold, and extracting the features of the depth image specifically comprises:

If the test mould is a folded surface mould, converting the depth image of the test mould into a pseudo-color image;

selecting a test area from the pseudo color image, and converting depth non-zero pixel points in the selected test area into three-dimensional coordinate points according to the parameters of the depth perception sensor;

For any three-dimensional coordinate point, acquiring a preset number of nearest neighbor points in the neighborhood of the three-dimensional coordinate point based on a KD-tree algorithm, performing plane fitting on the nearest neighbor points, acquiring a normal vector of the three-dimensional coordinate point according to a fitted plane, normalizing the normal vector, and acquiring a normal measurement value of the three-dimensional coordinate point;

selecting normal truth values of two adjacent planes of the test mold which are at right angles on a unit normal spherical surface;

Calculating Euclidean distances between the normal measurement value of the three-dimensional coordinate point and the two normal truth values respectively, and acquiring the minimum value of the two Euclidean distances;

And sequencing the minimum values corresponding to the three-dimensional coordinate points from small to large, calculating the weighted average distance between the normal measured value and the normal true value of the three-dimensional coordinate points according to the sequencing result, and taking the weighted average distance as the normal discrimination of the right-angled folding surface corresponding to the test mold.

9. The method of claim 8, wherein the weighted average distance between the normal measured value and the normal true value of the three-dimensional coordinate point is calculated according to the ranking result by:

and D is the weighted average distance, n is the number of the three-dimensional coordinate points, a is a preset proportion, alpha and beta are weights, di is the minimum value of the Euclidean distance between the normal measured value of the ith three-dimensional coordinate point in the sequencing result and the two normal truth values, and db is the Euclidean distance between the two normal truth values.

10. the depth image quality evaluation method based on face recognition application according to claim 2, wherein for any test mold, analyzing the depth image of the test mold, and extracting the features of the depth image specifically comprises:

if the test mould is a cylindrical surface mould, converting the depth image of the test mould into a pseudo-color image;

Selecting a test area from the pseudo color image, and converting depth non-zero pixel points in the selected test area into three-dimensional coordinate points according to the parameters of the depth perception sensor;

For any three-dimensional coordinate point, acquiring a preset number of nearest neighbor points in the neighborhood of the three-dimensional coordinate point based on a KD-tree algorithm, performing plane fitting on the nearest neighbor points, acquiring a normal vector of the three-dimensional coordinate point according to a fitted plane, normalizing the normal vector, and acquiring a normal measurement value of the three-dimensional coordinate point;

Projecting the normal measurement value of the three-dimensional coordinate point to an XY coordinate plane on a unit normal spherical surface to obtain a projection point of the normal measurement value of the three-dimensional coordinate point;

fitting projection points of normal measurement values of all the three-dimensional coordinate points into a straight line, and calculating the distance from each projection point to the straight line;

And calculating the proportion of the projection points with the distance smaller than the third preset threshold value in all the projection points, and taking the proportion as the normal smoothness of the cylindrical surface corresponding to the test mould.