WO2018049952A1

WO2018049952A1 - Photo acquisition method and device

Info

Publication number: WO2018049952A1
Application number: PCT/CN2017/096825
Authority: WO
Inventors: 刘守达
Original assignee: 厦门幻世网络科技有限公司
Priority date: 2016-09-14
Filing date: 2017-08-10
Publication date: 2018-03-22
Also published as: CN106503614A; CN106503614B

Abstract

A photo acquisition method, comprising: acquiring an initial image of a user (S101); acquiring a face model of the user according to the initial image (S102), the face model comprising facial feature points of the user; performing detection on the face model (S103); scoring the initial image according to a scoring standard on the basis of the detection result (S104), the scoring standard represents a corresponding relation between a detection result and a score value; and outputting a photo according to the scoring result of the initial image (S105). The method can further prompt the user to adjust the posture according to the detection result and/or the scoring result of the initial image. The method can further prompt the user to keep photographing when a particular condition is met, and acquire the initial image of the user according to a preset rule. Also disclosed is a photo acquisition device, comprising an acquisition module, a modeling module, a detection module, a scoring module, and an output module. By means of the technical solution, detection is performed on a photo of a user and the photo is scored, so that a photo conforming to a specified standard is acquired, and a foundation for rebuilding a 3D model is laid.

Description

Photo acquisition method and device

The present application claims the priority of the Chinese Patent Application, the entire disclosure of which is hereby incorporated by reference. .

Technical field

The present application relates to the field of image processing, and in particular, to a method and an apparatus for acquiring a photo.

Background technique

With the continuous development of modern science and technology, face recognition technology, as an emerging biometric technology in the development of digital information, has been widely used in many fields. For example, in key security units such as schools, airports, and courts, people can pass people. Face recognition technology implements access control or security monitoring; in Internet applications, face recognition technology can be used to establish its own biometric identification for payment and other online services with high security requirements; in entertainment games, people can pass the game. Face recognition technology builds its own 3D model to enhance the experience of users in entertainment games.

In various application scenarios of the face recognition technology, the face image is often obtained by taking a picture of the user, and the user often subconsciously makes some actions when the picture is taken, and the expression causes the face image to be irregular, for example, oblique The upper 45° angle shooting, the side face chin, the bangs blocking the forehead, etc., these irregular movements and expressions will seriously affect the recognition of the face image. Especially in entertainment games, if the user's photo does not meet the standard, the user's 3D model cannot be reconstructed based on the user's face photo. Therefore, how to obtain a standard photo that a user can reconstruct a 3D model has become an urgent problem to be solved.

Summary of the invention

In order to solve the above problem, the present application provides a photo acquisition method and device, which aims to obtain a photo that meets a specified standard for a user, and lays a foundation for reconstructing a 3D model.

The embodiment of the present application provides a photo acquisition method, including:

Obtain the initial image of the user;

Acquiring a face model of the user according to the initial image, where the face model includes a face feature point of the user;

Testing the face model;

And determining, according to the result of the detection, the initial image according to a scoring standard; wherein the scoring standard characterizes a correspondence between the detected result and the scoring score;

The photo is output according to the result of the rating of the initial image.

Optionally, in the photo acquiring method provided by the embodiment of the present application, after detecting the face model, the method further includes:

The user is prompted to adjust the posture based on the result of the detection and/or the result of the initial image.

The embodiment of the present application further provides a photo acquiring device, including:

An acquisition module, configured to acquire an initial image of the user;

a modeling module, configured to acquire a face model of the user according to the initial image, where the face model includes a face feature point of the user;

a detecting module, configured to detect the face model;

a scoring module, configured to score the initial image according to a scoring standard according to the result of the detecting; wherein the scoring standard characterizes a correspondence between the detected result and the scoring score;

And an output module, configured to output the photo according to the score result of the initial image.

The above at least one technical solution adopted by the embodiment of the present application can achieve the following beneficial effects:

(1) This application first obtains the initial image of the user, detects and scores the face model established based on the initial image, quantitatively evaluates the initial image of the user according to the scoring standard, and then evaluates the imaging quality according to the score result, and then outputs the specified image. Standard photo. This ensures that the acquired photos can meet the specified criteria and meet the needs of 3D model reconstruction, so that the user's 3D model can be reconstructed based on these photos.

(2) On the basis of detecting the obtained initial image of the user, the present application may also remind the user according to the result of detection and/or scoring when the user has an action and/or an expression that does not meet the specified standard. Adjust the posture to the standard action and/or expression, and then re-acquire the initial image after the user adjusts the posture, and further process, score, etc. the initial image. In this way, it is possible to help and guide the user to adjust their movements and/or expressions to obtain photos that meet the specified criteria.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the present application, and are intended to be a part of this application. In the drawing:

1 is a schematic flowchart of a method for acquiring a photo in an embodiment of the present application;

2 is a schematic flowchart of a method for acquiring a second photo in the embodiment of the present application;

FIG. 3 is a schematic structural diagram of a photo acquiring apparatus according to an embodiment of the present application.

detailed description

The technical solutions of the present application will be clearly and completely described in the following with reference to the specific embodiments of the present application and the corresponding drawings. It is apparent that the described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

The embodiment of the present application provides a photo acquisition method, as shown in FIG. 1 , including:

S101: Acquire an initial image of the user;

S102: Acquire a face model of the user according to the initial image, where the face model includes a face feature point of the user;

S103: detecting a face model;

S104: According to the result of the detection, the initial image is scored according to the scoring standard; wherein the scoring standard represents the correspondence between the detected result and the score score;

S105: Output a photo according to the score result of the initial image.

Each step in the above embodiment has multiple combinations of execution order and execution manner, for example:

(1) Each step can be sequentially performed in sequence. For example, after the initial image of one frame is obtained in step S101, the face model of the user is acquired according to the initial image of the frame, and then the face model is detected, scored, and then output according to the score result corresponding to the face model. Photo.

(2) One step or multiple steps in each step may be executed as needed after the loop is executed. For example, when the initial image is acquired in step S101, it may be executed a plurality of times to acquire a plurality of initial images. When performing steps S102 to S104, a plurality of face models may be separately processed to obtain a plurality of face models, and then each face model may be detected and scored. When testing and scoring, each item can be scored according to the test result of the item, or each item can be scored after detecting the entire item.

(3) For each initial image obtained, the processing may be performed by referring to the embodiment shown in FIG. 1. When a certain condition is met, the processing flow of an initial image may be ended, and the process returns to step S101 to reacquire. An initial image or a face model is identified, detected, and/or scored for the next initial image. For example, when the result of the scoring of the initial image does not reach the preset requirement, the photo may be output and the user may be informed of the score of the photo for the user to choose, or may directly jump back to step S101 to reacquire an initial image.

When the above steps are specifically implemented, the single thread may be sequentially performed (the steps may be sequentially performed sequentially or the steps may be performed in a certain step), or multiple threads may be used to execute multiple identical or different steps simultaneously (for example, multiple threads may be used) At the same time, a plurality of initial images are processed to obtain a face model, and multi-threads can also be used for acquiring a face model and detecting and/or scoring a face model, thereby completing multiple tasks simultaneously and improving resources. The efficiency of use can increase the efficiency of the system.

In the above embodiment, the initial image acquired in step S101 includes the user's face information and the photographing background information, and the initial image can be acquired by using a camera, a camera, or the like to acquire an image. When the initial image is acquired, it may be automatically acquired by the system or device according to the embodiment (for example, an initial image may be preset every 1 second or 1 frame), or may be adopted by the user through buttons, switches, and the like. Obtained after issuing an instruction to the system or apparatus employing the present embodiment.

Based on the initial image, various face recognition algorithms can be used to process the initial image to obtain the user's face model. Specifically, the face detector can detect the location of the face from the initial image, and the location of the face feature point is located in the region where the face is located using a random forest-based algorithm, thereby realizing the determination of the user's face based on the initial image. model. When selecting a face feature point, the number and position of the face feature points can be determined according to the implementation requirements. For example, 68 feature points or 77 feature points can be located and selected on the face model, and the relative feature can be further selected from the above. Feature points that do not change with the user's expression.

As an optional implementation manner, after obtaining the face model of the user, the face verification method based on Convolution Neural Networks (CNN) may be used to judge the sequence according to the region of the face feature point. Is the initial image and/or face model obtained from the same user: if yes, continue after execution The continuation step detects and scores the obtained face model; if not, returns to step S101 to reacquire the initial image of the user.

In the above embodiment, performing the step S103 to detect the face model may include detecting the occlusion, the brightness, the area of the eye, the area of the lips, and the motion state and/or the degree of clarity of the face model on the face model. Each test content is directed to a feature on the face model. For example, if the occlusion object on the face model is detected, it can be checked whether the face information of the user has been fully embodied in the face model, and whether there are obstructions such as hair, hats, scarves, and the like that may affect the integrity of the face information; The brightness of the face model is detected. It can be checked whether the illumination is sufficient when the user takes a picture, whether it is uniform, whether it will affect the extraction of face information; for the detection of the area of the eye on the face model, it is possible to check whether the user closes the eye when taking a picture. For the detection of the area where the lips are located, it is possible to check whether the user opens the mouth when taking a picture; to detect the motion state of the face model, it is possible to check whether the user takes a picture in a quiet state, whether a stable imaging effect can be formed; The degree of clarity is detected, and it is possible to check whether the initial image is blurred and whether the user's facial information can be clearly reflected. Each of the above detection contents has a certain emphasis. Therefore, when detecting according to the embodiment, one or more items of the items may be selected, and the quantity of the detected content and the order of detection are not affected; Performing multiple tests on a certain item, and performing the data processing result of the multiple detection results as the detection result of the content.

In the embodiment of the present application, after the face model is detected, the initial image is scored according to the detected result. At the time of scoring, the initial image can be scored according to a certain scoring standard, wherein the scoring standard characterizes the correspondence between the detected result and the scoring score. When scoring an initial image, the overall scoring results may be adjusted according to the detection results of different detection items, or each sub-scoring result may be scored separately for each detection item, and then multiple detection items are summarized. After the sub-scoring result, the overall scoring result of the initial image is obtained. In the summary, different weights can be further assigned to different detection items to reflect the influence of different detection items on the image quality and the degree of influence on the reconstruction of the 3D model. For example, the weight of the sub-score results of the occlusion detection can be set. The sub-scoring result is determined to be larger than the brightness detection, so that the sub-scoring result of the occlusion detection contributes more to the overall scoring result.

The specific detection process and scoring process for each test content will be illustrated item by item below.

When the step S103 is performed, the occlusion object on the face model may be detected, which may specifically include:

Calculating a first average color value of the entire face model; calculating a second average color value of the area to be tested on the face model;

A difference between the first average color value and the second average color value is calculated as a result of detecting whether the region to be tested has an occlusion.

When the occlusion on the face model is detected by the above method, one or more face regions that are easily occluded may be selected as the region to be tested, for example, a forehead that is easily blocked by the hair or the cap, and is easily blocked by the scarf. Chin, a nose mouth that is easily blocked by a mask, an eye that is easily blocked by an eye mask, and the like. Then calculating a first average color value for the entire area of the face model, and calculating a second average color value for the area to be tested, the difference between the first average color value and the second average color value can reflect the test Whether the area is occluded - if the area to be tested is not occluded, the color on the face model should be substantially uniform and there will be no excessive chromatic aberration. Therefore, when the score is scored according to the detection result, a difference threshold may be preset, and if the calculated difference between the first average color value and the second average color value is greater than the preset difference threshold, then I think there is an obstruction on the face model. Because the face shield will reduce the user's photo Quality, which has a negative impact on the reconstruction of the 3D model. Therefore, if the system assumes that the higher the score, the better the quality of the photo, then the detection of the obscuration on the face model should be deducted to reduce the score; otherwise, If the system assumes that the lower the score, the better the photo quality, it should be added when the face model is detected to have an occlusion to increase the score. For example, it can be specified in the scoring standard that if an obstruction is detected on the face model, the score score is directly deducted by 10 points to lower the score of the score.

More specifically, taking the forehead as the area to be tested as an example, when calculating the second average color value, the arithmetic mean value may be calculated, or the weighted average color value may be calculated according to the preset color weight as the second average color value. In view of the fact that it is more likely that an obstruction (such as a bangs) will appear on both sides of the area where the forehead is located (ie, directly above the eyebrows) than the middle part of the area where the forehead is located (ie, directly above the eyebrow), so The preset color weight of the two sides of the area where the forehead is located may be set higher than the preset color weight of the middle portion of the area where the forehead is located.

When performing step S103, the brightness on the face model may be detected, and specifically may include detecting one or more of the following:

Detecting whether the brightness on the face model is uniform;

Whether the brightness of the entire face model and/or the key area and the portion other than the face model on the initial image are uniform is detected.

More specifically, whether the brightness on the face model is uniform is detected, and by calculating the difference between the brightness values on both sides of the face model, it is possible to reflect whether the brightness of the person's face is uniform, that is, whether a "yin and yang face" phenomenon occurs. The specific calculation process can include:

Taking the central axis of the face model as a center line, symmetrically extracting the brightness values of the preset pair of pixels on both sides of the central axis;

The difference between the luminance values of the pairs of pixels symmetric with respect to the central axis is calculated as a detection result of detecting whether the luminance on the face model is uniform.

If the brightness of the face is uniform, the difference between the brightness values of the pairs of pixels symmetric with respect to the central axis should not be too large. Therefore, the brightness difference threshold can be set in advance, and the difference between the brightness values can be used to reach the preset brightness difference threshold. The number of pixels can be judged by determining whether the brightness of the person's face is uniform or not. The larger the above number, the more points indicating that the difference in luminance values in the symmetric pixel points is large, and the unevenness of the brightness on the human face, the greater the negative influence on the image quality. Therefore, if the system assumes that the higher the score score, the better the photo quality, the larger the number of pixels whose difference between the luminance values reaches the preset luminance difference threshold, the lower the score of the bonus points, or the score of the deduction. Should be higher; vice versa. For example, it can be specified in the scoring standard that if the number of pixels whose difference in luminance value reaches the preset luminance difference threshold is greater than 20, it is considered that there is a significant "yin and yang face" phenomenon, and 25 points are deducted from the score to reduce the score. The score; if the difference between the brightness values reaches the preset brightness difference threshold, the number of pixels is less than 20 and greater than 10, then a weak "yin and yang face" phenomenon is considered, and 10 points are deducted from the score to reduce the score.

In addition to the method of scoring the number of pixels in which the difference between the above statistical brightness values reaches the preset brightness difference threshold, the average value of the difference between the brightness values of the pair of pixel points may be used to score, wherein the average Values may include an arithmetic mean and/or a weighted average. The average value can reflect the average level of the brightness value difference of each pair of pixel points symmetric with respect to the central axis on the face model. The larger the average value, the larger the difference between the brightness values in the symmetric pixel points, the face of the person The more uneven the brightness, the greater the negative impact on image quality. Therefore, if the system assumes that the lower the score, the better the photo quality, the greater the average of the difference between the luminance values, the higher the score of the bonus points, or the points deducted. The lower the value should be; vice versa. The above average value may be an arithmetic mean value or a weighted average value, and when determining the weight, it may be considered from the viewpoint of the possibility of occurrence of the luminance difference and/or the influence on the imaging quality, for example, the possibility of occurrence of the luminance difference may be The weight of the pixel points of the larger face area is increased, and the weight of the pixel points of the face area having a large influence on the imaging quality can also be increased.

More specifically, when the brightness on the face model is detected, it is also possible to detect whether the brightness of the entire face model and/or the key area and the portion other than the face model on the initial image are uniform. By examining the difference between the brightness value of the face model and the brightness value of the image background, it can be judged whether the brightness of the entire initial image is uniform, and whether the face model part is insufficiently illuminated. This can be achieved in the following ways:

Calculating a first brightness average of the overall and/or key regions of the face model;

Calculating a second brightness average value of a portion other than the face model on the initial image;

The difference between the first brightness average value and the second brightness average value is calculated as a detection result of whether the brightness of the entire face model and/or the key area and the portion other than the face model on the initial image are uniform.

If the brightness of the face model and the portion other than the face model (ie, the image background) on the initial image is uniform, the difference between the first brightness average value and the second brightness average value should not be too large, so if the system assumes the score The higher the score, the better the photo quality, the greater the difference between the first brightness average and the second brightness average, the lower the score of the bonus points, or the higher the score of the deduction, and vice versa. .

When step S103 is performed, the area where the eye is located on the face model can be detected, including:

Determining the area of the eye on the face model according to the position of the face feature point;

Extracting a binary image of the area of the eye on the face model;

Positioning the upper and lower eyelids of the eye on the binary image;

The distance between the upper eyelid and the lower eyelid was calculated as a result of the test.

Before extracting the binary image of the area where the eye is located, the threshold area of the area where the eye is located may be firstly processed to screen out the information of the inner part of the eye. Extracting the binary image of the region where the eye is located, the contour information of the region where the eye is located can be extracted, thereby positioning the upper eyelid and the lower eyelid of the eye, and the distance between the upper eyelid and the lower eyelid is used as the detection result and the basis of the score. Specifically, a distance threshold may be preset. If the distance between the upper eyelid and the lower eyelid is less than the preset distance threshold, the user is considered to close the eye when taking a picture, and the image quality of the initial image acquired at this time does not meet the requirement. The reconstruction of the 3D model has a negative impact. Therefore, if the system assumes that the higher the score is, the better the photo quality is. If the closed eye is detected on the face model, the score should be deducted to reduce the score; otherwise, if the system assumes that the score is lower, the photo quality is higher. Well, it should be added when the closed eye is detected on the face model to increase the score. For example, it can be specified in the scoring standard that if the face model is detected to be closed, the score is directly deducted by 8 points to reduce the score of the score.

When step S103 is performed, the area where the upper lip is located on the face model can be detected, including:

Determining an inner lip of the upper lip, an inner edge of the lower lip, an outer edge of the upper lip, and an outer edge of the lower lip of the upper face of the face model according to the position of the feature point of the face;

A ratio of a first distance between the inner edge of the upper lip and the inner edge of the lower lip to a second distance between the outer edge of the upper lip and the outer edge of the lower lip is calculated as a result of the detection.

Specifically, 20 or so feature points can be selected in the area where the lips are located to determine the inner and outer edges of the upper and lower lips. By calculating the ratio of the first distance between the inner edge of the upper lip and the inner edge of the lower lip to the second distance between the outer edge of the upper lip and the outer edge of the lower lip, it is determined whether the user is open when taking a picture. In implementation, a lip ratio threshold may be preset. If the ratio of the first distance to the second distance is greater than the preset lip ratio threshold, the user may be considered to be open when taking a photo. Since the user's opening expression will have a negative impact on the reconstruction of the 3D model, if the system assumes that the higher the score score, the better the photo quality, the score on the face model should be deducted to reduce the score score; Conversely, if the system assumes that the lower the score, the better the photo quality, the score on the face model should be added to increase the score. For example, it can be specified in the scoring standard that if a face model opening is detected, 8 points are directly deducted from the score score to lower the score of the score.

When step S103 is performed, the motion state of the face model can be detected, including:

Calculating an interframe difference value of an initial image of the preset number of frames in which the face model is located;

It is determined whether the inter-frame difference value is smaller than the tenth preset value, and the result of the judgment is used as a result of the detection.

Specifically, the color value, the brightness value, or the inter-frame variation of each pixel of the two initial images separated by the preset number of frames may be extracted, and the difference between the two images, that is, the inter-frame difference value, is calculated. The preset number of preset frames can be taken as 1 frame, 5 frames, 10 frames, etc., and is not limited herein. If the user is not in motion when taking a picture, the inter-frame difference value should not be too large. Therefore, an inter-frame difference threshold (equivalent to the above-mentioned tenth preset value) can be preset as a measure, if the inter-frame difference value is smaller than the above. The inter-frame difference threshold indicates that the difference between the two initial images of the preset number of frames is small enough. It can be considered that the face model corresponding to the user is not in motion; otherwise, if the inter-frame difference value is greater than or equal to the inter-frame difference The threshold value indicates that the difference between the two initial images of the preset number of frames is large, and it can be considered that the face model corresponding to the user is in a motion state. Since the face model acquired when the user is in motion will have a negative impact on the reconstruction of the 3D model, if the system assumes that the higher the score score, the better the photo quality is, the buckle model should be deducted when it detects that the face model is in motion. To reduce the score score; conversely, if the system assumes that the score score is lower, the better the photo quality, then the face model should be added to increase the score when it is detected to be in motion. For example, it can be specified in the scoring standard that if the face model is detected to be in motion, 15 points are directly deducted from the score score to reduce the score of the score.

When step S103 is performed, the degree of clarity of the face model can be detected, including:

Calculating the variance of the initial image in which the face model is located;

It is judged whether the variance reaches the eleventh preset value, and the result of the judgment is taken as the result of the detection.

Specifically, the degree of clarity of the face model is determined by calculating the variance of the initial image. For the same image content, the larger the variance of the image, the clearer the image is. Therefore, the variance threshold (corresponding to the eleventh preset value above) can be set in advance as a measure if the variance is greater than or equal to the variance. The threshold value indicates that the image is clear enough to meet the specified criteria; if the variance is less than the variance threshold, the image is not clear enough and will have a negative impact on the reconstruction of the 3D model. Therefore, if the system assumes that the higher the score is, the better the photo quality is. If the face model is not clear enough, the score should be deducted to reduce the score. Conversely, if the system assumes that the score is lower, the photo quality is better. If the face model is not clear enough, it should be added to increase the score. For example, it can be specified in the scoring standard that if the face model is detected to be not clear enough, 20 points are directly deducted from the score score to reduce the score of the score.

The above movements on the face model, the brightness, the area of the eye, the area of the lips, and the movement of the face model The detection of the state and/or the degree of clarity can be performed for the face model of the plane, or the 3D model of the face model can be established according to the face feature points on the face model, and then the 3D model is detected. In particular, the deflection of the head portion of the 3D model in the three-dimensional direction can be detected in accordance with the 3D model, and in particular includes the angle of rotation of the head portion relative to the imaging device such as the camera/camera with respect to the X-axis, the Y-axis, and the Z-axis. In the specific implementation, the following methods can be used:

Determining a location of a stable point in a facial feature point according to the 3D model; the stable point is a feature point that changes only with the posture of the user's head;

Matching the preset head 3D model with the location of the stable point;

When the preset head 3D model matches the position of the stable point, the deflection angle of the preset head 3D model in the three-dimensional direction is extracted as the deflection angle of the head portion of the 3D model in the three-dimensional direction.

If the user does not face the imaging device such as a camera or a camera at the time of photographing, the face 3D model established based on the acquired initial image will have a deflection angle on the X-axis, the Y-axis, and/or the Z-axis, which will be a 3D model. Reconstruction has a negative impact, and the greater the deflection angle, the worse the image quality. Therefore, if the system assumes that the higher the score score, the better the photo quality, the larger the deflection angle, the lower the score of the bonus points, or the higher the score of the deduction points; if the system assumes that the score score is lower, The better the quality of the photo, the greater the deflection angle, the higher the score of the bonus points, or the lower the score of the deduction. For example, it can be specified in the scoring standard that if the deflection angle is 0 to 3°, the score is deducted by 0 to 5 points; when the deflection angle is 3 to 10 degrees, 5 to 20 points are deducted from the score. When the deflection angle is greater than 10°, the score of the score is deducted by 3 times the deflection angle. Deflection in different directions can also specify different deductions or bonus points in the scoring criteria.

In the foregoing embodiments, after performing one or more detections on the face model, the user may continue to perform the score according to the detection result, or may prompt the user to adjust the posture according to the detection result. For example, if the user is closed, the prompt is prompted. The user opens his eyes; if it detects that the user's head is deflected 30° to the right, the user is prompted to deflect 30° to the left, and the like. After scoring according to the detection result, the user may also be prompted to adjust the posture according to the scoring result. For example, if the user is deducted for the opening, the user may be prompted to close the mouth; if the user is detected that the forehead is blocked, the user is deducted. The user may be prompted to reveal the forehead; if it is detected that the user's score score does not meet the prescribed criteria, the user may be prompted to adjust the gesture. There are many ways to prompt the user to adjust the posture. You can choose one or a combination of voice, text, animation and other methods to guide the user to adjust to the standard, to obtain better test results or to get a better score. After prompting the user to adjust the gesture, the process returns to step S101 to reacquire the initial image after the user presets the preset number of frames. The preset number of frames referred to herein may be any preset value according to actual needs.

In each of the above embodiments, the scoring criterion for scoring based on the result of the detection may be a correspondence between the result of the detection established by the effect of the reference detection on the reconstruction of the 3D model and the score score. On this basis, the scoring standard can be adjusted and/or the scoring result can be adjusted according to the photographing duration and/or the scoring result of the initial image. When adjusting the scoring standard, the degree of reduction of the score based on the result of the test may be reduced, or the degree of increase of the score based on the result of the test may be increased. When adjusting the scoring result, the scoring result may be multiplied by a coefficient greater than 1, and a certain degree of amplification may be performed as an adjusted scoring result; or the scoring result may be multiplied by a coefficient less than 1, to be reduced to some extent as an adjustment. Post rating results.

More specifically, according to the duration of the photographing, it can be determined whether the photographing duration reaches a preset time (which can be recorded as the first preset time), and if the time is reached, the user can be considered to have spent a sufficient time (ie, a preset Time) Prepare for photographing, in the current situation it is difficult to obtain images that are more in line with the specified criteria, so that the criteria can be relaxed, the scoring criteria can be adjusted to make the scores higher or the points are less, and/or the method of directly amplifying the scoring results can be improved. The score of the initial image (the higher the score of the system is assumed to be, the better the photo quality is), so that the score of the initial image is more likely to reach the specified standard.

More specifically, based on the result of the scoring of the initial image, it may be determined whether the scoring result of the initial image does not reach a preset value (recorded as a first preset value), and if the first preset value is not reached, the user is represented. The score of the initial image is not high enough, and it can be considered that the initial image conforming to the standard has not been obtained (at this time, the higher the score of the system is, the better the photo quality is). At this time, the process may return to step S101 to reacquire the initial image. In some cases, for example, when the user subjectively approves the image, or objectively, the user has already prepared for a long enough time, the rating criteria may be actively adjusted so that the score is higher or the penalty is less, and / Or by directly magnifying the scoring result, the scoring score of the initial image is increased, so that the scoring score of the image is closer to the first preset value, and more likely to meet the specified standard.

More specifically, it may be determined whether the change value of the score result of the initial image is smaller than a preset value (recorded as a second preset value) according to the score result of the initial image, and if the change value of the score result is smaller than the second The preset value indicates that the initial image acquired by the user does not change much, and the improvement is not obvious. It can be considered that the user has not obtained the initial image with higher score score and can conform to the standard (the system assumes that the score score is higher) The better the quality of the photo, at this time, you can actively adjust the scoring standard so that the score is higher or the deduction is less, and/or the score of the initial image is improved by directly magnifying the scoring result, so that the score of the initial image is obtained. Scores are more likely to meet specified criteria.

In addition to checking whether the change value of the score result is too small, it is also possible to judge the change of the score of the initial image of the user by examining the statistics of the score result. For example, it may be determined whether the average value of the score results of the initial image of the preset number (recorded as the first preset number) reaches the third preset value, or may determine the preset number (recorded as the second preset number). Whether the standard deviation of the score result of the initial image reaches the fourth preset value. If it is not reached, it can be considered that the user has not obtained the initial image with higher score score and can meet the standard (the system assumes that the score score is higher) The better the quality of the photo, at this time, you can actively adjust the scoring standard so that the score is higher or the deduction is less, and/or the score of the initial image is improved by directly magnifying the scoring result, so that the score of the initial image is obtained. Scores are more likely to meet specified criteria.

In contrast to the above various adjustment methods for the scoring standard and/or the scoring result, the reference value of the above judgment may be adjusted, for example, the first preset value is lowered, so that the scoring score of the initial image is closer to the first A preset value is more likely to meet the specified criteria.

In the embodiment of the present application, referring to FIG. 2, after performing the step S104 according to the result of the detection, after scoring the initial image according to the scoring standard, before performing the step S105 according to the scoring result of the initial image, the present application The photo acquisition method also includes:

S106: Determine whether the score result of the initial image reaches a fifth preset value;

S107: If yes, the initial image and the result of the scoring are cached;

If not, return to step S101 to obtain the initial image of the user.

After scoring the initial image, it may be determined whether the score of the initial image (also referred to as a score score) reaches a preset score (remarked as a fifth preset value, and may also be regarded as a preset cache). Value) to judge the initial Whether the image meets the specified criteria for image caching. If the score score reaches the fifth preset value, the initial image may be considered to meet the cache standard, so the image conforming to the cache standard may be buffered for the user to select the output; if the score score does not reach the fifth preset value, It is considered that the initial image does not reach the specified cache standard, and the initial image can be discarded, and the process returns to step S101 to reacquire the initial image of the user. When the system caches the initial image, the image may be stored corresponding to the score of the score, or the detection result of the image and the image may be stored corresponding to the score score. It should be noted that a preset threshold (recorded as a sixth preset value) may be used to determine whether to output a photo. Specifically, it is determined whether the score of the initial image reaches a sixth preset value. Output photos. The values of the fifth preset value and the sixth preset value may be the same or different, that is, when the score result satisfies a certain condition (the condition indicates that the quality of the acquired initial image has reached the specified standard), the condition that satisfies the condition may be met. The image is directly output, or it can be cached first, and the optimal output is obtained after obtaining more images satisfying the condition.

When certain conditions are met, the fifth preset value and/or the sixth preset value may also be adjusted according to the photographing duration and/or the scoring result of the initial image. Specifically, the fifth preset value and/or the sixth preset value may be lowered or raised according to the value, the change value, the average value, and/or the standard deviation of the score result of the preset number of initial images. For example, when the photographing duration reaches the preset time, the scoring result of the initial image does not reach the preset value, the change value of the scoring result is less than the preset value, and the average value of the scoring result of the preset number of initial images does not reach the preset value, And/or the standard deviation of the scoring result of the preset number of initial images does not reach the preset value, the fifth preset value may be lowered, and the buffering of the standard may be implemented by reducing the cached standard, and the foregoing may also be reduced. Six preset values are selected by the user for the output of the photo by reducing the score standard of the output photo. If the user can obtain the initial image with a higher score after the fifth preset value or the sixth preset value is lowered, the preset value may be restored to the original level to obtain a better quality photo. In addition, if the scores of successive initial images reach the fifth preset value or the sixth preset value, the system can also increase the standard of the buffer or output, and raise the fifth preset value or the sixth preset. Set values to get better quality images. When the preset value (the fifth preset value and/or the sixth preset value) is adjusted to be raised or lowered, the multiplication by the coefficient may be adopted, or the method may be performed by increasing or decreasing a certain amplitude, for example, When the fifth preset value needs to be lowered, the fifth preset value may be multiplied by a coefficient smaller than 1, as the adjusted fifth preset value.

Further, after performing the step S104 according to the result of the detection, after the initial image is scored according to the scoring standard, before the photo is output according to the scoring result of the initial image in step S105, the user may be prompted when a certain condition is met. Keep taking photos and get the user's initial image according to the preset rules. The condition required to be met may be one or more of the following conditions:

The number of initial images acquired has reached the fifth preset number;

The average value of the scoring results of the initial image reaches a seventh preset value;

The standard deviation of the scoring result of the initial image reaches an eighth preset value;

The lowest score of the score result of the initial image reaches the ninth preset value.

The step of prompting the user to keep taking a picture may be performed after the initial image is cached in step S107, and the initial image of the cached image may be directly considered by the cache step of S107. Whether you can prompt the user to keep taking photos.

Among the items listed above, the fifth preset number represents an upper limit value of the number of initial images preset by the system, and may be the number of images stored in the cache, or may be the number of initial images acquired by the system. . Get the beginning The number of initial images has reached the fifth preset number, and it can be considered that the system has acquired enough images to meet certain criteria. The seventh preset value is used to check whether the average value of the score result reaches a preset value, and the eighth preset value is used to check whether the standard deviation of the score result reaches a preset limit value, and the ninth preset value is used for examination. Whether the lowest score of the score result reaches the preset limit value, and the satisfaction of any condition or multiple conditions indicates that the score of the initial image obtained by the system has reached a certain standard (the average value reaches the seventh preset value, the standard The difference reaches the eighth preset value, and/or the lowest score reaches the ninth preset value, indicating that the quality of the initial image acquired by the system has met certain requirements. In this case, the image can be directly output for the user to select, or the user can be prompted to keep taking a picture, and the initial image of the user is obtained according to a preset rule. It can be considered that, when the user obtains the initial image that satisfies certain requirements, the user will better maintain the posture after receiving the prompt to keep the photograph, thereby obtaining a better image.

After the user is prompted to take a photo, acquiring the initial image of the user according to the preset rule may include: acquiring the initial image of the user according to the preset interval frame number, the preset interval time, the preset image number, and/or the preset acquisition time. The preset number of images represents the total number of images acquired after prompting the user to keep taking a photo. The preset acquisition time indicates a continuous photographing time after prompting the user to keep taking a photo. The preset interval frame number indicates that the user is prompted to keep taking a photo after acquiring the photo. The number of frames spaced between two adjacent images. The preset interval time indicates the time interval between two adjacent images acquired after prompting the user to keep taking a picture. For example, the rule can be preset to obtain an initial image every 1 second (preset interval time), and a total of 10 (preset image number) initial images can be obtained, or the rule can be preset to be 5 minutes (pre-pre Let an initial image be acquired every 5 frames (preset interval frame number) within the acquisition time.

For the initial image obtained after the above prompts are kept photographed, detection and scoring can be performed. For a certain standard, the image can be directly output for the user to choose, can be cached, or can be directly discarded without caching (for example, detecting the image). And the scoring is used to monitor whether the user's posture during the photographing phase is stable and whether there is a change, and there is no need to cache the image. In determining whether to output or cache, the scoring results may be examined using the aforementioned criteria, and different criteria may be used. When the initial image is cached, the image may be directly stored, or the initial image with the lowest score in the original cache may be deleted, and then the initial image obtained by prompting the user to keep taking the photo may be stored.

After prompting the user to keep taking a picture, the user may also determine whether to continue to acquire the initial image of the user according to the preset rule according to the detection result and/or the score result of the initial image obtained after the user is prompted to take the picture. By monitoring and judging the detection result of the initial image and/or the scoring result, it is determined whether the user always maintains a posture sufficient to satisfy the requirement after receiving the prompt to keep the photograph. For example, if the obtained consecutive multiple initial images do not meet the requirements of output or buffer, it can be considered that the user has not satisfied the condition for maintaining the photographing, so the image should not be continuously acquired according to the preset rule, and the process proceeds to step S101 to obtain the initial image. . For example, when it is detected that the user's head is deflected or closed, and the posture of the user does not meet the preset criteria, the image should not be continuously acquired according to the preset rule, and the process proceeds to step S101 to obtain the initial image. In this way, the user image that prompts the user to keep taking a picture is monitored (monitoring with a preset interval frame number and/or a preset interval time), which is beneficial to ensure that the user is prompted to keep the initial user after taking the picture. The image quality of the image.

The prompt for keeping the photo taken by the user may be prompted by a combination of any one or more of voice, text, image, animation, and the like. It can be considered that the user will pay more attention to maintaining the posture after getting the prompt. The initial image acquired at this stage should be of better quality and the score should be higher (the system assumes that the higher the score, the better the photo quality), therefore, the user is prompted. The original image of the user acquired after taking the photo can be adjusted to the result of the initial image. The adjusted score result is used as the score result of the initial image. Specifically, the score result may be multiplied by a coefficient greater than 1 or less than 1, as an adjusted score result.

In the present application, when the photo is output, all the cached photos may be output to the user in a certain order according to the scoring result of the initial image, or a photo that reaches a certain score or other conditions may be output in real time, or may be in the following manner. get on:

Screening the initial image based on the results of the test;

The preset number of photos with the highest score results are output based on the scored results of the filtered initial images.

When screening the initial image based on the results of the detection, a number of different ways can be employed. For example, screening can be performed according to strict conditions. For example, in the result of head detection, the face deflection angle should not be greater than 3°, and closed eyes or openings should not occur. The images that meet the requirements are sorted according to the score results, and finally Output the preset number of photos with the highest score; if all the photos are not satisfactory when screening according to strict conditions, you can relax the conditions for screening, for example, adjust the head deflection angle to no more than 3° to no more than 10°, then Then sort the output according to the score result for the user to select. Correspondingly, it is also possible to gradually tighten the requirements according to the first loose and strict screening methods, and screen out photos that meet the standards. When screening the initial image according to the detection result, it is also possible to prioritize the detection result of the detection item with greater influence on the image quality according to the difference degree of the influence of different detection items on the image quality, and directly screen the initial image with serious defects. In addition to (even if the score is higher, it is also screened out). For example, the occlusion of the user's face has a great influence on the image quality. Therefore, the result of the occlusion detection can be first screened as the initial image without the occlusion, and then filtered according to other detection items. When the photo with the highest score is output, the scoring result may be the overall scoring result of the initial image, or may be the scoring result of a certain detection item, which may be selected according to different requirements of the photo, and is not limited herein. After the user gets the photo that meets the standard, the 3D model can be reconstructed based on the photo to meet the application requirements. For example, the overall score of image 1 is 88 points, the obstruction detection reaches 95 points, the brightness detection reaches 80 points, the overall score of image 2 is 90 points, the obstruction detection reaches 91 points, and the brightness detection reaches 88 points. For the reconstruction of the 3D model, there is no obstruction than the brightness, so in the output, the image 1 will be given priority for the user to select.

It should be noted that, in each of the foregoing embodiments, the specific values of the preset time, the preset number, the preset value, the preset threshold, the preset frame, the preset frame number, and the preset buffer value may be the same, Can be different. For example, the first preset value, the fifth preset value, and the sixth preset value may be the same or different.

The present application also provides a photo acquisition device, as shown in FIG. 3, comprising:

The obtaining module 101 is configured to acquire an initial image of the user;

The modeling module 102 is configured to acquire a face model of the user according to the initial image, where the face model includes a face feature point of the user;

a detecting module 103, configured to detect a face model;

The scoring module 104 is configured to score the initial image according to the scoring standard according to the result of the detecting; wherein the scoring standard represents a correspondence between the detected result and the scoring score;

The output module 105 is configured to output a photo according to the score result of the initial image.

The above apparatus may further include:

The posture adjustment prompting module is configured to prompt the user to adjust the posture according to the detected result and/or the score result of the initial image;

a caching module, configured to cache an initial image and a scoring result when the scoring result of the initial image reaches a preset cache value;

The prompting photo module is configured to prompt the user to keep taking a photo when the preset condition is met.

The above-mentioned photo acquisition device corresponds to the description of the flow of the photo acquisition method described above, and the deficiencies refer to the description of the above method flow, and will not be further described.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory. Memory is an example of a computer readable medium.

Computer readable media includes both permanent and non-persistent, removable and non-removable media. Information storage can be implemented by any method or technology. The information can be computer readable instructions, data structures, modules of programs, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), Dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, only Read compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape storage or other magnetic storage device or any other non-transportable medium that can be used for storage can be calculated Information accessed by the device. As defined herein, computer readable media does not include temporary storage of computer readable media, such as modulated data signals and carrier waves.

It is also to be understood that the terms "comprises" or "comprising" or "comprising" or any other variations are intended to encompass a non-exclusive inclusion, such that a process, method, article, Other elements not explicitly listed, or elements that are inherent to such a process, method, commodity, or equipment. An element defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device including the element.

Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system, or computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment in combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The above description is only an embodiment of the present application and is not intended to limit the application. Various changes and modifications can be made to the present application by those skilled in the art. Any modifications, equivalents, improvements, etc. made within the spirit and scope of the present application are intended to be included within the scope of the appended claims.

Claims

A photo acquisition method, comprising:

Obtain the initial image of the user;

Acquiring a face model of the user according to the initial image, where the face model includes a face feature point of the user;

Testing the face model;

And determining, according to the result of the detection, the initial image according to a scoring standard; wherein the scoring standard characterizes a correspondence between the detected result and the scoring score;

The photo is output according to the result of the rating of the initial image.
The method according to claim 1, wherein after detecting the face model, the method further comprises:

The user is prompted to adjust the posture based on the result of the detection and/or the result of the initial image.
The method of claim 1 wherein the method further comprises:

The scoring criteria are adjusted and/or adjusted based on the duration of the photographing and/or the scoring results of the initial image.
The method according to claim 3, wherein the scoring criteria are adjusted and/or adjusted according to the photographing duration and/or the scoring result of the initial image, including:

Adjusting and/or adjusting the rating criteria if one or more of the following conditions are met:

The photographing duration reaches a first preset time;

The score of the initial image does not reach the first preset value;

The change value of the score result of the initial image is smaller than the second preset value;

The average of the score results of the first preset number of initial images does not reach the third preset value;

The standard deviation of the scoring results of the second predetermined number of initial images does not reach the fourth preset value.
The method of claim 4 wherein the rating criteria are adjusted to include:

The degree of decrease in the score score based on the result of the test is reduced, or the degree of increase in the score score based on the result of the test is increased.
The method of claim 4, wherein the adjusting the score results comprises:

The score result is multiplied by a coefficient greater than 1 or less than 1, as an adjusted score result.
The method according to claim 1, wherein after the initial image is scored according to a scoring standard according to the result of the detection, the method is further processed before the photo is output according to the scoring result of the initial image. include:

If the result of the scoring of the initial image reaches a fifth preset value, the initial image and its scoring result are cached.
The method according to claim 1, wherein the outputting the photo according to the result of the scoring of the initial image comprises:

If the result of the scoring of the initial image reaches a sixth preset value, the photo is output.
The method according to claim 7 or 8, wherein after the initial image is scored according to the scoring standard according to the result of the detecting, the method further comprises:

The fifth preset value and/or the sixth preset value are adjusted according to the photographing duration and/or the scoring result of the initial image.
The method according to claim 9, wherein the adjusting the fifth preset value and/or the sixth preset value according to the result of scoring the initial image comprises:

The fifth preset value and/or the sixth preset value are lowered or increased according to a value, a change value, an average value, and/or a standard deviation of a score result of a preset number of initial images.
The method according to claim 1, wherein after the initial image is scored according to the scoring standard according to the result of the detecting, before the outputting the photo according to the scoring result of the initial image, the method further comprises:

If any of the following conditions are met, the user is prompted to take a photo and obtain the initial image of the user according to a preset rule:

The number of the obtained initial images has reached a fifth preset number;

The average value of the scoring results of the initial image reaches a seventh preset value;

The standard deviation of the score result of the initial image reaches an eighth preset value;

The lowest score of the score result of the initial image reaches a ninth preset value.
The method according to claim 11, wherein the initial image of the user is obtained according to a preset rule, including:

The initial image of the user is obtained according to the preset interval frame number, the preset interval time, the preset image number, and/or the preset acquisition time.
The method according to claim 11, wherein the prompting the user to keep taking a photo and acquiring the initial image of the user according to the preset rule comprises:

The initial image of the user is determined to be continued according to the preset rule according to the detection result and/or the scoring result of the initial image obtained after the user is prompted to take the photo.
A method according to any one of claims 11 to 13, wherein

For the initial image of the user that is obtained after the user is prompted to take a photo, the scoring result of the initial image is adjusted, and the adjusted scoring result is used as the scoring result of the initial image.
The method according to claim 1, wherein the detecting of the face model comprises: obstructing, brightness, area of the eye, area of the lips, movement of the face model on the face model State and/or clarity are tested.
The method according to claim 1, wherein after the capturing of the face model of the user according to the initial image, before detecting the face model, the method further comprises:

Establishing a 3D model of the face model according to the face feature point on the face model;

The detecting the face model specifically includes:

According to the 3D model, the deflection of the head portion of the 3D model in a three-dimensional direction is detected.
The method according to claim 16, wherein detecting the deflection of the head portion of the 3D model in a three-dimensional direction according to the 3D model comprises:

Determining, according to the 3D model, a location of a stable point in the facial feature point; the stable point is a feature point that changes only with a posture of a user's head;

Matching the preset head 3D model with the location of the stable point;

When the preset head 3D model matches the position of the stable point, extracting a deflection angle of the preset head 3D model in a three-dimensional direction as a head portion of the 3D model in a three-dimensional direction Deflection angle.
The method according to claim 1, wherein the outputting the photo according to the result of the scoring of the initial image comprises:

Filtering the initial image according to the result of the detection;

The preset number of photos with the highest score result is output according to the scored result of the filtered initial image.
A photo acquisition device, comprising:

An acquisition module, configured to acquire an initial image of the user;

a modeling module, configured to acquire a face model of the user according to the initial image, where the face model includes a face feature point of the user;

a detecting module, configured to detect the face model;

a scoring module, configured to score the initial image according to a scoring standard according to the result of the detecting; wherein the scoring standard characterizes a correspondence between the detected result and the scoring score;

And an output module, configured to output the photo according to the score result of the initial image.
The device according to claim 19, wherein said device further comprises:

a posture adjustment prompting module, configured to prompt the user to adjust the posture according to the result of the detection and/or the result of the initial image;

a caching module, configured to cache the initial image and the scoring result when the scoring result of the initial image reaches a preset cache value;

The prompting photo module is configured to prompt the user to keep taking a photo when the preset condition is met.