WO2018188007A1 - 自拍的方法、装置和终端设备 - Google Patents
自拍的方法、装置和终端设备 Download PDFInfo
- Publication number
- WO2018188007A1 WO2018188007A1 PCT/CN2017/080335 CN2017080335W WO2018188007A1 WO 2018188007 A1 WO2018188007 A1 WO 2018188007A1 CN 2017080335 W CN2017080335 W CN 2017080335W WO 2018188007 A1 WO2018188007 A1 WO 2018188007A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- target image
- target
- image
- determining
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
Definitions
- the present application relates to the field of photographing, and more particularly to a method, apparatus and terminal device for self-photographing in the field of photographing.
- the application provides a self-timer method and device, which can reduce the probability of invalid photo shooting when the user takes a selfie, and reduce the system energy consumption while improving the user experience.
- a method for self-timer comprising: acquiring a first target image and a second target image, the first target image and the second target image being faces of a first target user acquired at different times Determining, according to the first target image and the second target image, whether the facial expression of the first target user is in a stable state; when the facial expression of the first target user is in a stable state, determining the first target image or Whether a smile is included in the second target image; when the first target image or the second target image includes a smile, the camera shutter is controlled to take a picture through the camera.
- the self-photographing method determines whether the facial expression of the user is in a stable state by first determining whether the facial expression of the user is in a stable state, and then determining whether the facial expression of the user includes a smiling face at this time, and finally determines Under the premise that the user's facial expression is the most brilliant state of the smile, the camera shutter is automatically controlled to take pictures through the camera, and the image with the most smile of the user is recorded. Avoid controlling the shooting of invalid photos caused by the camera shutter through the camera based on whether the user's facial expression includes a smile, that is, reducing the probability of invalid photo shooting, and improving the user experience.
- the self-photographing method provided by the present application determines whether to perform smile detection according to the judgment result of the user's expression steady state, and when the user's expression is not in a stable state, no complicated smile detection is needed, thereby greatly reducing system energy consumption. .
- determining, according to the first target image and the second target image, whether the facial expression of the first target user is in a stable state including: calculating the first a similarity between the target image and the second target image; when the similarity is greater than or equal to the preset first threshold, determining that the facial expression of the first target user is in a stable state.
- determining whether the first target image or the second target image includes a smiley face including: when the first target user's face When the expression is in a steady state, and the steady state is the first occurrence, it is determined whether the smile is included in the first target image or the second target image.
- a time interval between an acquisition time of the first target image and an acquisition time of the second target image is less than or equal to a preset
- the second threshold is greater than or equal to the preset third threshold.
- the time interval between the acquisition time of the first target image and the acquisition time of the second target image is less than or equal to a preset second threshold, and is greater than or equal to a preset third threshold, so that the mobile device is not wasted Computational resources, without missing the smile saturation period of the user's facial expressions, improving the user experience.
- the method further includes: determining, by the multi-user self-portrait mode, the first target user from the at least two users.
- the determining the first target user from the at least two users includes: determining a user closest to the camera as the first target a user; or a user who has the largest number of pixels occupied by the face image as the first target user; or a user corresponding to the user operation as the first target user according to a user operation; or an image stored in the memory The corresponding user is determined to be the first target user.
- the first target image and the second target image are mouth images covering the same facial range of the first target user.
- a self-photographing apparatus comprising: an acquiring unit, configured to acquire a first target image and a second target image, wherein the first target image and the second target image are acquired at different times a facial image of the target user; determining unit, configured to determine, according to the first target image and the second target image, whether the facial expression of the first target user is in a stable state; the determining unit is further configured to: when the first Determining whether a smile is included in the first target image or the second target image when the facial expression of the target user is in a stable state; and the control unit is configured to control when the smile is included in the first target image or the second target image
- the camera shutter takes a picture through the camera.
- the camera shutter is automatically controlled to take pictures through the camera, recording the most brilliant image of the user. Avoid controlling the shooting of invalid photos caused by the camera shutter through the camera based on whether the user's facial expression includes a smile, that is, reducing the probability of invalid photo shooting, and improving the user experience.
- the self-photographing method provided by the present application determines whether to perform smile detection according to the judgment result of the user's expression steady state, and when the user's expression is not in a stable state, no complicated smile detection is needed, thereby greatly reducing system energy consumption. .
- the determining unit is specifically configured to: calculate a similarity between the first target image and the second target image; the determining unit is further configured to: When the similarity is greater than Or equal to the preset first threshold, determining that the facial expression of the first target user is in a stable state.
- the determining unit is further configured to: when the facial expression of the first target user is in a stable state, and the stable state is first When it occurs, it is determined whether a smile is included in the first target image or the second target image.
- the time interval between the acquiring time of the first target image and the acquiring time of the second target image is less than or equal to a preset
- the second threshold is greater than or equal to the preset third threshold.
- the time interval between the acquisition time of the first target image and the acquisition time of the second target image is less than or equal to a preset second threshold, and is greater than or equal to a preset third threshold, so that the mobile device is not wasted Computational resources, without missing the smile saturation period of the user's facial expressions, improving the user experience.
- the determining unit is further configured to determine the first target user from the at least two users for the multi-user self-portrait mode.
- the determining unit is specifically configured to: determine a user closest to the camera as the first target user; or occupy a pixel of the facial image The user having the largest number is determined as the first target user; or determining the user corresponding to the user operation as the first target user according to the user operation; or determining the user corresponding to the image stored in the memory as the first target user.
- the first target image and the second target image are mouth images covering the same facial range of the first target user.
- a terminal device comprising: a memory, a processor and a camera; the memory is configured to store an instruction; the processor is configured to invoke an instruction in the memory to perform the following steps: the processor, And acquiring the first target image and the second target image, the first target image and the second target image are facial images of the first target user acquired at different times; the processor is configured to use the first target image and Determining, by the second target image, whether the facial expression of the first target user is in a stable state; the processor is further configured to: when the facial expression of the first target user is in a stable state, determine the first target image or the first Whether the smile is included in the two target images; the processor is further configured to control the camera shutter to take a photo through the camera when the first target image or the second target image includes a smile.
- the camera shutter is automatically controlled to take pictures through the camera, recording the most brilliant image of the user. Avoid controlling the shooting of invalid photos caused by the camera shutter through the camera based on whether the user's facial expression includes a smile, that is, reducing the probability of invalid photo shooting, and improving the user experience.
- the self-photographing method provided by the present application determines whether to perform smile detection according to the judgment result of the user's expression steady state, and when the user's expression is not in a stable state, no complicated smile detection is needed, thereby greatly reducing system energy consumption. .
- the processor is specifically configured to: calculate a similarity between the first target image and the second target image; the processor is further configured to: When the similarity is greater than or equal When the preset first threshold is used, it is determined that the facial expression of the first target user is in a stable state.
- the processor is specifically configured to: when the facial expression of the first target user is in a stable state, and the stable state is the first time When present, it is determined whether a smile is included in the first target image or the second target image.
- the time interval between the acquiring time of the first target image and the acquiring time of the second target image is less than or equal to a preset
- the second threshold is greater than or equal to the preset third threshold.
- the time interval between the acquisition time of the first target image and the acquisition time of the second target image is less than or equal to a preset second threshold, and is greater than or equal to a preset third threshold, so that the mobile device is not wasted Computational resources, without missing the smile saturation period of the user's facial expressions, improving the user experience.
- the processor is further configured to determine the first target user from the at least two users for the multi-user self-portrait mode.
- the processor is specifically configured to: determine a user closest to the camera as the first target user; or use a pixel occupied by the facial image The user having the largest number is determined as the first target user; or determining the user corresponding to the user operation as the first target user according to the user operation; or determining the user corresponding to the image stored in the memory as the first target user.
- the first target image and the second target image are mouth images covering the same facial range of the first target user.
- a computer storage medium stores program code for indicating execution of any of the optional self-timer methods of performing the above first aspect or the first aspect Operation.
- FIG. 1 is a schematic flow chart of a method of self-timer according to the present application.
- FIG. 2 is a schematic diagram showing feature point matching of a first target image and a second target image.
- Figure 3 shows a schematic diagram of a mouth image of a first target user of the same range and the same central position.
- FIG. 4 is a schematic block diagram of a self-timer device in accordance with the present application.
- FIG. 5 is a schematic structural diagram of a self-timer terminal device according to the present application.
- the technical solution of the present application aims to automatically control the camera shutter to take a picture through the camera at the time when the most smile of the user is detected.
- FIG. 1 shows a schematic flow chart of a method 200 of a self-timer according to an embodiment of the invention. As shown in FIG. 1, the method 100 includes:
- the mobile phone device can acquire the current user image, for example, acquiring the first target image and the second target image of the user, where the first target image and the second target image are the first target users acquired at different times. Facial image.
- the mobile phone device After the mobile phone device acquires the first target image and the second target image, determining, by the first target image and the second target image, whether the facial expression of the first target user is in a stable state, when the first target user When the facial expression is in a steady state, it is determined whether a smile is included in the first target image or the second target image.
- the camera shutter of the mobile phone device is automatically activated to take a photo for the first target user, thereby achieving the purpose of intelligent self-timer.
- steady state means that the facial expression of the user reaches a stable state, and the facial muscle of the user is basically maintained at the same position, that is, when the facial expression of the user reaches a steady state, the facial image of the user is over time.
- the magnitude of the change is small.
- the self-photographing method of the present application by first determining whether the facial expression of the user is in a stable state, and determining that the facial expression of the user is in a stable state, determining whether the facial expression of the user at this time includes a smiling face (ie, determining the user's The facial expression satisfies both the steady state and the smiling condition, and finally determines the user's facial expression as the most brilliant state of the smile, and then automatically controls the camera shutter to take a picture through the camera, recording the most brilliant image of the user. Avoid controlling the shooting of invalid photos caused by the camera shutter through the camera based on whether the user's facial expression includes a smile, that is, reducing the probability of invalid photo shooting, and improving the user experience. And determining whether to perform smile detection according to the judgment result of the user's expression steady state, when the user's expression is not in a stable state, complicated smile detection is not required, thereby greatly reducing system energy consumption.
- determining whether the facial expression of the first target user is in a stable state according to the first target image and the second target image comprises: calculating a similarity between the first target image and the second target image And determining that the facial expression of the first target user is in a stable state when the similarity is greater than or equal to a preset first threshold.
- the similarity between the first target image and the second target image may be calculated.
- the calculated similarity is greater than or equal to the preset first threshold, it is determined that the facial expression of the first target user is in a stable state.
- the first target image and the second target image are mouth images covering the same facial range of the first target user.
- the user when the user turns on the front camera of the mobile phone device to perform self-photographing, the user adjusts the distance and angle between the user and the front camera of the mobile phone device, so that the entire facial image of the user can appear on the display screen of the mobile phone device. in. Therefore, the first target image and the second target image acquired by the mobile phone device often include a complete facial image of the user.
- the feature is mainly passed
- the mouth organ in the facial organ of the user is embodied. Therefore, in order to save computational resources, the acquired first target image and the second target image are processed, that is, the first target image and the second target image are intercepted, and the first A mouth image of the target user calculates a similarity between the mouth image corresponding to the first target image and the mouth image corresponding to the second target image.
- the user may move back and forth, left and right.
- the corresponding facial image occupies a larger number of pixels than the corresponding facial image when the user is away from the front camera.
- the face detection algorithm for example, the Haar cascade algorithm
- the first target image and the second target image actually cover the user's face range and image.
- the position of the face corresponding to the center may also be different.
- the first target image and the second target image are required to include the same range of the first target user and the mouth image of the same center position. Therefore, it is first necessary to determine the mouth image of the same range and the same center position in the first target image and the second target image.
- the acquired first target image and the second target image are intercepted, that is, the first target image and the second target image are intercepted from the middle position, and the first target image and the second target image are obtained.
- the lower half facial image, the lower half facial image corresponding to the first target image is denoted as I 1 (as shown by the image on the left in FIG. 2 ), and the lower half facial image corresponding to the second target image is denoted as I 2 (As shown in the image on the right in Figure 2).
- the feature point sets V 1 and V 2 in the images I 1 and I 2 are determined by the Speeded Up Robust Features (SURF) algorithm, and the feature point sets V 1 and V 2 are matched to obtain a feature point matching pair.
- M (V 1 , V 2 ).
- the feature value of each pixel in the image I 1 and the image I 2 is calculated and quantized by the SURF algorithm, and the feature values of each pixel point are judged one by one, when the feature value is greater than the preset feature value threshold. Then, the corresponding feature value is taken as the candidate feature point, and finally N candidate feature points with the larger feature value are taken as the final feature point.
- the position of the N final feature points in the user's face image is indicated by a small circle in FIG.
- All final matching feature points in each feature point and the final image is calculated by one image I I, 1 2, when a final feature points in the images I 1 2 in a final feature the matching image I Degree is the maximum matching degree of the matching degree between the final feature point in the image I 1 and all the final feature points in the image I 2 , and when the matching degree is greater than the preset matching degree threshold, the maximum matching degree
- the final feature point in the corresponding image I 1 and the final feature point in the image I 2 are a pair of feature point matching pairs.
- the two feature points connected by a straight line in FIG. 2 are a pair of feature point matching pairs, and the other feature points not connected by a straight line are feature points that have no matching success (ie, no match is found to be greater than the preset match). Characteristic point of the degree threshold).
- the two feature points are connected by a straight line in FIG. 2 only for the purpose of more intuitively displaying the matching result of the feature points, and the actual feature point matching process does not exist in the step.
- the matching pair in the image I 1 and the image I 2 is determined, and the method may have a matching error when determining the matching pair.
- the preset matching is not suitable for the threshold value, it may cause the determination result of the matching pair to be erroneous. Therefore, it is necessary to correct the matching pair determined by the above method (for example, culling the matching pair of matching errors).
- the feature point matching pairs determined thereon represent the distribution of the same face position of the first target user on image I 1 and on image I 2 . Therefore, it is impossible for two feature points in the same matching pair to generate too much fluctuation between the distribution of the image I 1 and the image I 2 (for example, the feature point of the user's left face image is impossible to be compared with the user's right face image) The feature points in the match match successfully).
- the distance between the pixel positions corresponding to the two feature points in the same matching pair in the image I 1 and the image I 2 can be calculated, and the distance is compared with the distance threshold of the preset pixel position, if the same match If the distance between the pixel positions corresponding to the two feature points of the pair is less than or equal to the distance between the pixel positions corresponding to the two feature points in the same matching pair, the feature point matching pair is an incorrect matching pair, and the pair The error feature points match the culling, otherwise they are retained.
- the error feature point matching pair is coarsely filtered by the above method for culling the error feature point matching pair, and the feature point matching pair retained after the coarse filtering is recorded as M(V 1 ', V 2 '), and the image I 1 is calculated respectively.
- a mean point of the feature point retained in the image I 2 wherein the mean point of the feature point of the image I 1 is denoted as C 1 , and the mean point of the feature point of the image I 2 is denoted as C 2 (image I 1 and the image I 2 is the average point of the image I 1 and I 2 is the center point of the image).
- the image I 1 and the image I 2 mismatch pair are finely filtered by the method of limiting the mean square variance.
- the For loop is used to calculate the mean value of the feature point displacement, and the feature point matching pair whose displacement deviates from the displacement point of the feature point is found as the suspect matching pair. If the deviation point of the feature point of the suspect matching pair is greater than the preset displacement offset, When the value is thresholded, the suspect match point is removed. Then continue to use the for loop to calculate the displacement mean and find the deviation matching pair until all the feature point matching pairs are offset from the displacement average value by less than the preset displacement offset value threshold.
- the fine-filtered feature point matching pair M(V 1 ′, V 2 _adjusted') is used to perform displacement and amplification correction on the image I 1 and the image I 2 , and finally accurately obtain the same coverage in the image I 1 and the image I 2 .
- mouth image area shown in Figure 3. wherein the left image in Figure 3 the mouth portion corresponding to the image I 1 as shown, the image shown in the right side of the mouth portion 3 image I 2 corresponding to FIG.
- the image I 1 and the image I 2 already include the same range and the same center position.
- the mouth image of the first target user as shown in Figure 3.
- the ratio is less than or equal to the preset ratio threshold, the similarity between the representative image I 1 and the image I 2 is high, and the similarity between the image I 1 and the image I 2 is further determined, and when the similarity is greater than Or equal to the preset first threshold, indicating that the facial expression of the first target user is in a stable state.
- the above description only takes a method of determining the facial expression of the user by calculating the similarity, but the present application is not limited thereto. May also be (e.g., by determining the position of the image I 1 and the moving unit 2 of the face image I, and to compare the displacement of the moving unit 1 and the I picture image I 2) determines whether a user's facial expression by the facial modeling method It is in a stable state. Alternatively, it is also possible to determine whether the facial expression of the user is in a stable period by extracting facial feature points and then comparing the displacement of the facial feature points in the image I 1 and the image I 2 . This application does not limit this.
- the next step can be made to determine whether the facial expression of the user includes a smiling face.
- the mobile device detects that the user's facial expression includes a smiling face, the mobile device camera shutter is automatically activated to take a picture for the user.
- the entire facial image or a partial facial image of the user may be applied by a feature extraction method of a local binary pattern (LBP) or a feature extraction method in other image processing (for example, Feature extraction of the lower half of the user's face image, and then the extracted feature points are sent to the trained classifier (for example, the classifier may be a Support Vector Machine (SVM)), and the face image is determined. Whether to include a smiley image.
- SVM Support Vector Machine
- other smile detection methods can also be used to determine whether the user's facial image includes a smiley image.
- the user's facial image can be sent to a trained artificial neural network model (for example, in a Convolutional Neural Network (CNN)).
- CNN Convolutional Neural Network
- the shutter is automatically activated to take a picture.
- multiple photos may be taken continuously.
- the judgment condition is increased. For example, for the current self-timer, it is determined whether the steady state is the first time, and only when the steady state is first appeared in the current self-timer. It is detected whether the user's facial expression includes a smile.
- step S110 the time interval between the acquisition time of the first target image and the acquisition time of the second target image is less than or equal to a preset second threshold, and is greater than or equal to a preset third threshold.
- the time interval is set to be very short (for example, the time interval is set to 20 milliseconds)
- the user's facial expression may only occur very small within the 20 milliseconds.
- the change for example, the user's mouth key point only has a slight displacement.
- the system since the user's facial expression only changes little, it may cause the system to misjudge, that is, the system may think that the small change is caused by system noise (for example, the system misjudges the small change)
- the change caused by the shaking of the mobile phone device during the photographing process is not caused by the user's facial expression changing during the time interval, and thus the system cannot accurately judge whether the facial expression of the user is in a stable state;
- the time interval cannot be set too long.
- the time interval is set too long (for example, the time interval is set to 500 milliseconds)
- the system may miss the user's facial expression.
- the smile saturation period causes the system to not take pictures of the user during the user's smile saturation period.
- the technical solution provided by the present application sets the time interval within an interval, that is, the time interval is small. And equal to or equal to a preset second threshold, and greater than or equal to a preset third threshold. Therefore, it is ensured that the system can analyze whether the facial expression of the user is in a stable state with a sufficiently high frequency, and does not cause the system to misjudge because the setting of the time interval is too small, occupying the computing resources of the mobile device; and it is not because of the time. Setting the interval too long causes the system to miss the user's smile saturation period, thus ensuring that the user's experience is improved and the system energy consumption is reduced.
- the value of the time interval should also satisfy the condition that the image acquisition period of the camera of the mobile phone device is an integral multiple of the image capturing period, for example, the camera captures the image refresh.
- the rate is 50 frames per second, that is, an image is captured every 20 milliseconds, and the image acquisition period of the camera is 20 milliseconds. Therefore, the time interval should be an integer multiple of 20 milliseconds (for example, the time interval is 20 milliseconds, 40 milliseconds, 60 milliseconds, etc., which are not enumerated here).
- the value of the time interval will also fluctuate up and down (that is, the value of the time interval will correspondingly become larger) Or become smaller).
- the value of the time interval can be appropriately reduced;
- the value of the time interval can be appropriately increased at this time.
- the accuracy of the overall system is improved and the power consumption is reduced by changing the value of the time interval in real time and flexibly according to the distance of the user's face from the camera.
- the value of the first threshold corresponding to the similarity used to determine whether the facial expression of the user is in a stable state changes according to the change of the value of the time interval.
- the first threshold corresponding to the degree is correspondingly set to be larger, so as to avoid false determination due to whether the first facial value is set small, and whether the facial expression of the user is in a stable state;
- the time interval is set to be large, the degree of change of the facial expression of the user is also large during the time interval, and accordingly, the similarity of the facial expression of the user in the time interval is lower, and therefore,
- the first threshold corresponding to the similarity should also be set to be correspondingly small to avoid erroneous judgment as to whether the facial expression of the user is in a stable state due to the large setting of the first threshold.
- the above determining whether the facial expression of the first target user is in a stable state is for the single-person self-portrait mode.
- the method for determining the first target user in the multiplayer self-timer mode will be described below for the multiplayer self-timer mode.
- the mobile device group needs to determine the first target user from at least two users.
- one of the plurality of people may be determined as a first target user who subsequently determines whether the facial expression is in a stable state.
- the mobile phone device may determine the user closest to the camera as the first target user; or determine the user whose face image has the largest number of pixels among the plurality of people as the first target user; or according to the user operation, the user The user corresponding to the operation is determined as the first target user.
- the avatar performs selection of the first target user; or the user corresponding to the image stored in the memory is determined as the first target user, for example, when multiple users simultaneously appear in the same display screen, among the multiple users
- the avatar of a certain user is used as the avatar of the user in the address book, and the mobile device can determine the first target user according to the avatar of the user saved in the address book, that is, the user corresponding to the avatar from the plurality of users. Determined as the first target user.
- the self-photographing method of the present application by first determining whether the facial expression of the user is in a stable state, and determining that the facial expression of the user is in a stable state, determining whether the facial expression of the user includes a smiling face at this time. Finally, under the premise of determining that the user's facial expression is the most brilliant state of the smile, the camera shutter is automatically controlled to take a picture through the camera, and the image with the most smile of the user is recorded. Avoid controlling the shooting of invalid photos caused by the camera shutter through the camera based on whether the user's facial expression includes a smile, that is, reducing the probability of invalid photo shooting, and improving the user experience.
- FIG. 4 shows a schematic block diagram of a self-photographing device 200 in accordance with the present application.
- the apparatus 200 includes an acquisition unit 210, a determination unit 220, and a control unit 230.
- the acquiring unit 210 is configured to acquire a first target image and a second target image, where the first target image and the second target image are facial images of the first target user acquired at different times;
- a determining unit 220 configured to determine, according to the first target image and the second target image, whether the facial expression of the first target user is in a stable state
- the determining unit 220 is further configured to:
- the control unit 230 is configured to control the camera shutter to take a photo through the camera when the first target image or the second target image includes a smile.
- the determining unit 220 is specifically configured to:
- the determining unit 220 is further specifically configured to:
- the determining unit 220 is further configured to:
- the facial expression of the first target user is in a stable state, and the stable state is the first occurrence, it is determined whether the smile is included in the first target image or the second target image.
- the time interval between the acquisition time of the first target image and the acquisition time of the second target image is less than or equal to a preset second threshold, and is greater than or equal to a preset third threshold.
- the determining unit 220 is further configured to:
- the first target user is determined from at least two users for the multi-user self-portrait mode.
- the determining unit 220 is specifically configured to:
- a user corresponding to the image stored in the memory is determined as the first target user.
- the first target image and the second target image are mouth images covering the same facial range of the first target user.
- the apparatus 200 for self-timer according to the present application may correspond to an implementation body of the method 100 for self-timer of the present application, and the units in the apparatus 200 for self-timer and the other operations and/or functions described above are respectively implemented
- the corresponding process of the method 100 in FIG. 1 is not repeated here for brevity.
- the apparatus for self-timer of the present application by first determining whether the facial expression of the user is in a stable state, On the basis of determining that the facial expression of the user is in a stable state, it is determined whether the facial expression of the user includes a smiling face at this time, and finally, under the premise that the facial expression of the user is determined to be the most brilliant state, the camera shutter is automatically controlled by the camera. Take a picture and record the most brilliant image of the user. Avoid controlling the shooting of invalid photos caused by the camera shutter through the camera based on whether the user's facial expression includes a smile, that is, reducing the probability of invalid photo shooting, and improving the user experience. And determining whether to perform smile detection according to the judgment result of the user's expression steady state, when the user's expression is not in a stable state, complicated smile detection is not required, thereby greatly reducing system energy consumption.
- FIG. 5 shows a schematic block diagram of a terminal device 300 according to the present application.
- the terminal device 300 includes a processor 310, a memory 320, and a camera 330.
- the memory 320 is used to store instructions
- the processor 310 is configured to execute instructions stored in the memory 320 to control the camera shutter to take pictures through the camera 330.
- the memory 320 may include a volatile memory such as a random-access memory (RAM); the memory may also include a non-volatile memory such as a flash memory.
- RAM random-access memory
- non-volatile memory such as a flash memory.
- HDD hard disk drive
- SSD solid-state drive
- the memory 320 may also include a combination of the above types of memories.
- the processor 310 can be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP.
- the processor 310 may further include a hardware chip.
- the hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof.
- the PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a general array logic (GAL), or any combination thereof.
- the processor 310 is configured to acquire a first target image and a second target image, where the first target image and the second target image are facial images of the first target user acquired at different times;
- the processor 310 is further configured to determine, according to the first target image and the second target image, whether the facial expression of the first target user is in a stable state;
- the processor 310 is further configured to determine whether a smile is included in the first target image or the second target image when the facial expression of the first target user is in a stable state;
- the processor 310 is further configured to control the camera shutter to take a photo through the camera 330 when the first target image or the second target image includes a smile.
- the processor 310 is specifically configured to:
- the processor 310 is further specifically configured to:
- the processor 310 is specifically configured to:
- the facial expression of the first target user is in a stable state, and the stable state is the first occurrence, it is determined whether the smile is included in the first target image or the second target image.
- the time interval between the acquisition time of the first target image and the acquisition time of the second target image is less than or equal to the preset second threshold and is greater than or equal to the preset third threshold.
- processor 310 is further configured to:
- the first target user is determined from at least two users for the multi-user self-portrait mode.
- the processor 310 is specifically configured to:
- a user corresponding to the image stored in the memory is determined as the first target user.
- the first target image and the second target image are mouth images covering the same facial range of the first target user.
- the terminal device 300 may correspond to an implementation body of the method 100 for self-timer of the present application, and each unit in the terminal device 300 and the other operations and/or functions described above are respectively implemented to implement the method 100 of FIG.
- the corresponding process for the sake of brevity, will not be described here.
- the terminal device for self-timer of the present application by first determining whether the facial expression of the user is in a stable state, on the basis of determining that the facial expression of the user is in a stable state, determining whether the facial expression of the user at this time includes a smiling face, and finally Under the premise that the user's facial expression is the most brilliant state of the smile, the camera shutter is automatically controlled to take a picture through the camera, and the image with the most smile of the user is recorded. Avoid controlling the shooting of invalid photos caused by the camera shutter through the camera based on whether the user's facial expression includes a smile, that is, reducing the probability of invalid photo shooting, and improving the user experience. And determining whether to perform smile detection according to the judgment result of the user's expression steady state, when the user's expression is not in a stable state, complicated smile detection is not required, thereby greatly reducing system energy consumption.
- the size of the serial numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not be implemented in the present application.
- the process constitutes any limitation.
- the disclosed systems, devices, and methods may be implemented in other manners.
- the device embodiments described above are merely illustrative.
- the division of the unit is only a logical function division.
- there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
- the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to implement the solution of the embodiment. purpose.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
- the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product.
- the technical solution of the present application which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including
- the instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present application.
- the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .
Abstract
本申请提供了一种自拍的方法和装置，该方法包括：获取第一目标图像和第二目标图像，该第一目标图像与该第二目标图像为在不同时刻获取的第一目标用户的面部图像；根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态；当该第一目标用户的面部表情处于稳定状态时，确定该第一目标图像或该第二目标图像中是否包括笑脸；当该第一目标图像或该第二目标图像中包括笑脸时，控制相机快门通过摄像头进行拍照，通过在确定用户的面部表情为笑容最灿烂的状态的前提下，再自动控制相机快门通过摄像头进行拍照，记录用户笑容最灿烂的图像，降低了无效照片拍摄的概率，并且改善了用户体验。
Description
本申请涉及拍照领域，并且更具体地，涉及拍照领域中自拍的方法、装置和终端设备。
随着智能手机的不断普及，其具有的拍照功能也受到越来越多用户的喜爱。尤其是年轻女性，她们使用前置摄像头自拍的频率甚至超过了使用后置摄像头的频率。
目前，随着音量调节键拍照、延迟拍照、自拍杆等功能和设备的出现，拍照操作的复杂度在很大程度上得到了简化，但是仍然不够智能，仍然需要用户手动操作或者配合计时器来摆出表情。
生活中可以发现，人们在拍照时，笑容在所有表情中出现的几率最大，通过检测笑容表情的人脸图像，可以实现基于笑脸检测的自动激活快门的自拍功能。
然而，针对基于笑脸检测的自动自拍功能，目前很多手机厂商都在进行研究，但效果并不理想。主要问题在于无法准确地在用户笑容最饱满的时候进行拍照，即，由于自动激活快门的时机不恰当，导致错过用户笑容最饱满的拍照时机，影响用户体验。
发明内容
本申请提供了一种自拍的方法和装置，能够在用户自拍时降低无效照片拍摄的概率，并在改善用户体验的前提下降低系统能耗。
第一方面，提供了一种自拍的方法，该方法包括：获取第一目标图像和第二目标图像，该第一目标图像与该第二目标图像为在不同时刻获取的第一目标用户的面部图像；根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态；当该第一目标用户的面部表情处于稳定状态时，确定该第一目标图像或该第二目标图像中是否包括笑脸；当该第一目标图像或该第二目标图像中包括笑脸时，控制相机快门通过摄像头进行拍照。
因此，本申请提供的自拍的方法，通过首先确定用户的面部表情是否处于稳定状态，在确定用户的面部表情处于稳定状态的基础上，再确定此时用户的面部表情是否包括笑脸，最终在确定用户的面部表情为笑容最灿烂的状态的前提下，再自动控制相机快门通过摄像头进行拍照，记录用户笑容最灿烂的图像。避免仅仅根据用户的面部表情是否包括笑脸来控制相机快门通过摄像头所导致的对无效照片的拍摄，即降低了无效照片拍摄的概率，并且改善了用户体验。
并且，本申请提供的自拍的方法，根据用户表情稳定状态的判断结果，确定是否进行笑脸检测，当用户表情不处于稳定状态时，则不需要再进行复杂的笑脸检测，从而大幅降低系统能耗。
结合第一方面，在第一方面的第一种实现方式中，该根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态，包括：计算该第一目标图像与该第二目标图像之间的相似度；当该相似度大于或等于预设的第一阈值时，确定该第一目标用户的面部表情处于稳定状态。
结合第一方面及其上述实现方式，在第一方面的第二种实现方式中，该确定该第一目标图像或该第二目标图像中是否包括笑脸，包括：当该第一目标用户的面部表情处于稳定状态，且该稳定状态为第一次出现时，确定该第一目标图像或该第二目标图像中是否包括笑脸。
通过增加判断条件，即，对于当前自拍，判断该稳定状态是不是第一次出现，只有当该稳定状态为在当前自拍中首次出现时，才检测用户的面部表情是否包括笑脸。避免对笑容饱和期持续时间较长的用户连续拍照多张相同的照片，即节约了拍照设备的存储资源，又提升了用户体验。
结合第一方面及其上述实现方式，在第一方面的第三种实现方式中，该第一目标图像的获取时刻与该第二目标图像的获取时刻之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
通过使第一目标图像的获取时间与第二目标图像的获取时间之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值，使得既不浪费手机设备的计算资源，又不会错过用户面部表情的笑容饱和期，改善用户体验。
结合第一方面及其上述实现方式，在第一方面的第四种实现方式中，该方法还包括：针对多用户自拍模式，从至少两个用户中确定该第一目标用户。
结合第一方面及其上述实现方式，在第一方面的第五种实现方式中，该从至少两个用户中确定该第一目标用户，包括：将距离摄像头最近的用户确定为该第一目标用户；或将面部图像所占像素数最多的用户确定为该第一目标用户；或根据用户操作，将与该用户操作对应的用户确定为该第一目标用户；或将与存储器中存储的图像对应的用户确定为该第一目标用户。
结合第一方面及其上述实现方式，在第一方面的第六种实现方式中，该第一目标图像与该第二目标图像为覆盖该第一目标用户的相同面部范围的嘴部图像。
第二方面，提供了一种自拍的装置，该装置包括：获取单元，用于获取第一目标图像和第二目标图像，该第一目标图像与该第二目标图像为在不同时刻获取的第一目标用户的面部图像；确定单元，用于根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态；该确定单元还用于：当该第一目标用户的面部表情处于稳定状态时，确定该第一目标图像或该第二目标图像中是否包括笑脸；控制单元，用于当该第一目标图像或该第二目标图像中包括笑脸时，控制相机快门通过摄像头进行拍照。
通过首先确定用户的面部表情是否处于稳定状态，在确定用户的面部表情处于稳定状态的基础上，再确定此时用户的面部表情是否包括笑脸，最终在确定用户的面部表情为笑容最灿烂的状态的前提下，再自动控制相机快门通过摄像头进行拍照，记录用户笑容最灿烂的图像。避免仅仅根据用户的面部表情是否包括笑脸来控制相机快门通过摄像头所导致的对无效照片的拍摄，即降低了无效照片拍摄的概率，并且改善了用户体验。
并且，本申请提供的自拍的方法，根据用户表情稳定状态的判断结果，确定是否进行笑脸检测，当用户表情不处于稳定状态时，则不需要再进行复杂的笑脸检测，从而大幅降低系统能耗。
结合第二方面，在第二方面的第一种实现方式中，该确定单元具体用于：计算该第一目标图像与该第二目标图像之间的相似度；该确定单元具体还用于：当该相似度大于
或等于预设的第一阈值时，确定该第一目标用户的面部表情处于稳定状态。
结合第二方面及其上述实现方式，在第二方面的第二种实现方式中，该确定单元具体还用于：当该第一目标用户的面部表情处于稳定状态，且该稳定状态为第一次出现时，确定该第一目标图像或该第二目标图像中是否包括笑脸。
通过增加判断条件，即，对于当前自拍，判断该稳定状态是不是第一次出现，只有当该稳定状态为在当前自拍中首次出现时，才检测用户的面部表情是否包括笑脸。避免对笑容饱和期持续时间较长的用户连续拍照多张相同的照片，即节约了拍照设备的存储资源，又提升了用户体验。
结合第二方面及其上述实现方式，在第二方面的第三种实现方式中，该第一目标图像的获取时刻与该第二目标图像的获取时刻之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
通过使第一目标图像的获取时间与第二目标图像的获取时间之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值，使得既不浪费手机设备的计算资源，又不会错过用户面部表情的笑容饱和期，改善用户体验。
结合第二方面及其上述实现方式，在第二方面的第四种实现方式中，该确定单元还用于：针对多用户自拍模式，从至少两个用户中确定该第一目标用户。
结合第二方面及其上述实现方式，在第二方面的第五种实现方式中，该确定单元具体用于：将距离摄像头最近的用户确定为该第一目标用户；或将面部图像所占像素数最多的用户确定为该第一目标用户；或根据用户操作，将与该用户操作对应的用户确定为该第一目标用户；或将与存储器中存储的图像对应的用户确定为该第一目标用户。
结合第二方面及其上述实现方式，在第二方面的第六种实现方式中，该第一目标图像与该第二目标图像为覆盖该第一目标用户的相同面部范围的嘴部图像。
第三方面，提供了一种终端设备，其特征在于，包括：存储器，处理器和摄像头；该存储器用于存储指令；该处理器用于调用该存储器中的指令执行以下步骤：该处理器，用于获取第一目标图像和第二目标图像，该第一目标图像与该第二目标图像为在不同时刻获取的第一目标用户的面部图像；该处理器，用于根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态；该处理器，还用于当该第一目标用户的面部表情处于稳定状态时，确定该第一目标图像或该第二目标图像中是否包括笑脸；该处理器，还用于当该第一目标图像或该第二目标图像中包括笑脸时，控制相机快门通过该摄像头进行拍照。
通过首先确定用户的面部表情是否处于稳定状态，在确定用户的面部表情处于稳定状态的基础上，再确定此时用户的面部表情是否包括笑脸，最终在确定用户的面部表情为笑容最灿烂的状态的前提下，再自动控制相机快门通过摄像头进行拍照，记录用户笑容最灿烂的图像。避免仅仅根据用户的面部表情是否包括笑脸来控制相机快门通过摄像头所导致的对无效照片的拍摄，即降低了无效照片拍摄的概率，并且改善了用户体验。
并且，本申请提供的自拍的方法，根据用户表情稳定状态的判断结果，确定是否进行笑脸检测，当用户表情不处于稳定状态时，则不需要再进行复杂的笑脸检测，从而大幅降低系统能耗。
结合第三方面，在第三方面的第一种实现方式中，该处理器具体用于：计算该第一目标图像与该第二目标图像之间的相似度；该处理器具体还用于：当该相似度大于或等
于预设的第一阈值时，确定该第一目标用户的面部表情处于稳定状态。
结合第三方面及其上述实现方式，在第三方面的第二种实现方式中，该处理器具体用于：当该第一目标用户的面部表情处于稳定状态，且该稳定状态为第一次出现时，确定该第一目标图像或该第二目标图像中是否包括笑脸。
通过增加判断条件，即，对于当前自拍，判断该稳定状态是不是第一次出现，只有当该稳定状态为在当前自拍中首次出现时，才检测用户的面部表情是否包括笑脸。避免对笑容饱和期持续时间较长的用户连续拍照多张相同的照片，即节约了拍照设备的存储资源，又提升了用户体验。
结合第三方面及其上述实现方式，在第三方面的第三种实现方式中，该第一目标图像的获取时刻与该第二目标图像的获取时刻之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
通过使第一目标图像的获取时间与第二目标图像的获取时间之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值，使得既不浪费手机设备的计算资源，又不会错过用户面部表情的笑容饱和期，改善用户体验。
结合第三方面及其上述实现方式，在第三方面的第四种实现方式中，该处理器还用于：针对多用户自拍模式，从至少两个用户中确定该第一目标用户。
结合第三方面及其上述实现方式，在第三方面的第五种实现方式中，该处理器具体用于：将距离摄像头最近的用户确定为该第一目标用户；或将面部图像所占像素数最多的用户确定为该第一目标用户；或根据用户操作，将与该用户操作对应的用户确定为该第一目标用户；或将与存储器中存储的图像对应的用户确定为该第一目标用户。
结合第三方面及其上述实现方式，在第三方面的第六种实现方式中，该第一目标图像与该第二目标图像为覆盖该第一目标用户的相同面部范围的嘴部图像。
第四方面，提供了一种计算机存储介质，所述计算机存储介质中存储有程序代码，所述程序代码用于指示执行上述第一方面或第一方面的任意可选的实现自拍的方法所执行的操作。
图1是根据本申请的自拍的方法的示意性流程图。
图2示出了第一目标图像和第二目标图像进行特征点匹配的示意图。
图3示出了相同范围和相同中心位置的第一目标用户的嘴部图像的示意图。
图4是根据本申请的自拍的装置的示意性框图。
图5是根据本申请的自拍的终端设备的示意性结构图。
本申请的技术方案旨在实现在检测到用户笑容最灿烂的时刻自动控制相机快门通过摄像头进行拍照。
图1示出了根据本发明实施例的自拍的方法200的示意性流程图，如图1所示，该方法100包括：
S110，获取第一目标图像和第二目标图像，该第一目标图像与该第二目标图像为在不同时刻获取的第一目标用户的面部图像；
S120，根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态；
S130，当该第一目标用户的面部表情处于稳定状态时，确定该第一目标图像或该第二目标图像中是否包括笑脸；
S140，当该第一目标图像或该第二目标图像中包括笑脸时，控制相机快门通过摄像头进行拍照。
下面以用户通过支持自拍功能的手机设备自拍为例，对本申请的技术方案进行说明。
具体而言，当用户(例如，第一目标用户)开启手机设备的摄像头进行自拍时，用户的面部表情就会被实时地显示在手机屏幕当中。此时，手机设备可以对当前的用户图像进行获取，例如，获取用户的第一目标图像和第二目标图像，该第一目标图像与该第二目标图像为在不同时刻获取的第一目标用户的面部图像。
当手机设备获取该第一目标图像和该第二目标图像之后，通过该第一目标图像和该第二目标图像，确定此时第一目标用户的面部表情是否处于稳定状态，当第一目标用户的面部表情处于稳定状态时，确定该第一目标图像或该第二目标图像中是否包括笑脸。
当手机设备检测到该第一目标图像或该第二目标图像中包括笑脸时，则手机设备的拍照快门自动激活，为第一目标用户进行拍照，从而达到智能自拍的目的。
需要说明的是，上述“稳定状态”是指用户的面部表情达到稳定状态，此时用户的面部肌肉基本维持位置不变，即，当用户的面部表情达到稳定状态时，用户的面部图像随时间的变化幅度很小。
根据本申请的自拍的方法，通过首先确定用户的面部表情是否处于稳定状态，在确定用户的面部表情处于稳定状态的基础上，再确定此时用户的面部表情是否包括笑脸(即，确定用户的面部表情同时满足处于稳定状态与包括笑脸两个条件)，最终在确定用户的面部表情为笑容最灿烂的状态的前提下，再自动控制相机快门通过摄像头进行拍照，记录用户笑容最灿烂的图像。避免仅仅根据用户的面部表情是否包括笑脸来控制相机快门通过摄像头所导致的对无效照片的拍摄，即降低了无效照片拍摄的概率，并且改善了用户体验。并且根据用户表情稳定状态的判断结果，确定是否进行笑脸检测，当用户表情不处于稳定状态时，则不需要再进行复杂的笑脸检测，从而大幅降低系统能耗。
可选地，该根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态，包括：计算该第一目标图像与该第二目标图像之间的相似度；当该相似度大于或等于预设的第一阈值时，确定该第一目标用户的面部表情处于稳定状态。
具体而言，在确定第一目标用户的面部表情是否处于稳定状态时，可以通过计算该第一目标图像和该第二目标图像之间的相似度。当计算得到的相似度大于或者等于预设的第一阈值时，则确定该第一目标用户的面部表情处于稳定状态。
可选地，该第一目标图像与该第二目标图像为覆盖该第一目标用户的相同面部范围的嘴部图像。
具体而言，当用户打开手机设备的前置摄像头进行自拍时，用户会通过调节自身与手机设备的前置摄像头的距离及角度，以使用户的整个面部图像都能出现在手机设备的显示屏幕中。因此，手机设备获取的该第一目标图像和该第二目标图像往往包括用户的完整的面部图像。
当用户的面部表情处于笑容饱和期(例如，稳定状态的一例)时，该特征主要通过
用户的面部器官中的嘴部器官体现，因此，为了节约计算资源，对获取的第一目标图像和第二目标图像进行处理，即，将第一目标图像和第二目标图像进行截取，获取第一目标用户的嘴部图像，计算该第一目标图像对应的嘴部图像与该第二目标图像对应的嘴部图像之间的相似度。
然而，用户通过手机设备的前置摄像头进行自拍时，用户有可能会前后、左右移动。用户靠近前置摄像头时对应的面部图像所占的像素数量大于用户远离前置摄像头时对应的面部图像所占的像素数量。
此外，由于人脸检测算法(例如，Haar cascade算法)的检测精度存在误差，即使用户保持与摄像头相对位置稳定不变，该第一目标图像与该第二目标图像实际覆盖的用户面部范围以及图像中心对应的面部位置也有可能不同。
在计算用户嘴部表情的相似度时，需要第一目标图像与该第二目标图像包括第一目标用户的相同范围和相同中心位置的嘴部图像。因此，首先需要确定第一目标图像与第二目标图像中的相同范围和相同中心位置的嘴部图像。
下面对本申请中的确定第一目标图像与第二目标图像中的相同范围和相同中心位置的嘴部图像的方法进行说明。
首先对获取的第一目标图像与第二目标图像进行截取，即，从该第一目标图像与该第二目标图像的正中间位置进行截取，获得该第一目标图像与该第二目标图像的下半部面部图像，将第一目标图像对应的下半部面部图像记为I1(如图2中左边的图像所示)，将第二目标图像对应的下半部面部图像记为I2(如图2中右边的图像所示)。
通过加速稳健特征(Speeded Up Robust Features，SURF)算法确定图像I1、I2中的特征点集合V1、V2，并对该特征点集合V1、V2进行匹配，获得特征点匹配对M(V1，V2)。
具体而言，通过SURF算法计算并量化图像I1和图像I2中的每个像素点的特征值，并对每个像素点的特征值逐一进行判断，当特征值大于预设的特征值阈值时，则将对应的特征值取为候选特征点，最后将特征值较大的N个候选特征点作为最终的特征点。将该N个最终的特征点在用户面部图像中的位置在图2中采用小圆圈进行标示。
逐一计算图像I1中的每个最终特征点与图像I2中的所有最终特征点的匹配度，当图像I1中的某个最终特征点与图像I2中的某个最终特征点的匹配度是图像I1中的该最终特征点与图像I2中的所有最终特征点的匹配度中最大的匹配度时，且当该匹配度大于预设的匹配度阈值时，则该最大匹配度对应的图像I1中的最终特征点与图像I2中的最终特征点为一对特征点匹配对。图2中通过直线连接的两个特征点为一对特征点匹配对，其他没有通过直线连接的特征点为没有匹配成功的特征点(即，没有找到与其之间的匹配度大于预设的匹配度阈值的特征点)。
需要说明的是，图2中通过直线将两个特征点进行连接，目的只是为了更直观地显示特征点的匹配结果，实际的特征点匹配过程中并不存在该步骤。
然而，通过计算特征点之间的匹配度且确定该匹配度是否大于预设的匹配度阈值来确定图像I1与图像I2中的匹配对，该方法在确定匹配对时有可能出现匹配错误(例如，当预设的匹配对阈值的取值不合适时，可能会导致匹配对的确定结果出错)。因此，有必要对通过上述方法确定的匹配对进行校正(例如，剔除匹配错误的匹配对)。
为了剔除错误匹配的匹配对。对于图像I1与图像I2而言，在其上确定的特征点匹配对代表的是第一目标用户的相同的面部位置在图像I1和在图像I2上的分布情况。因此，
同一匹配对中的两个特征点在图像I1与图像I2中的分布情况之间不可能中产生太大的波动(例如，用户左脸图像的特征点不可能与用户右脸图像中的特征点匹配成功)。
因此，可以计算图像I1与图像I2中的同一匹配对中的两个特征点对应的像素位置之间的距离，再将该距离与预设的像素位置的距离阈值进行比较，若同一匹配对中的两个特征点对应的像素位置之间的距离小于或等于该同一匹配对中的两个特征点对应的像素位置之间的距离，则该特征点匹配对为错误匹配对，将该错误特征点匹配对剔除，否则保留。
采用上述剔除错误特征点匹配对的方法对错误特征点匹配对进行了粗过滤，将经过粗过滤后保留的特征点匹配对记为M(V1’，V2’)，分别计算图像I1与图像I2中保留的特征点的均值点，其中，将图像I1的特征点的均值点记为C1，将将图像I2的特征点的均值点记为C2(图像I1与图像I2的均值点即为图像I1与图像I2的中心点)。再根据M(V1，V2)、C1与C2，通过公式Z＝Σ(V′1-C1)/Σ(V′2-C2)计算图像I1相对于图像I2的放大系数Z。
计算均值点C1与C2的坐标偏差，根据该坐标偏差与Z，获取矫正后的特征点集合V2中每个点的坐标分别相对于特征点集合V1上匹配的点的位置，即V2_adjusted。
对于特征点匹配对M(V1’,V2_adjusted)，采用限制平均值方差的方法对图像I1与图像I2错误匹配对进行细过滤。例如，采用for循环计算特征点位移平均值，并找到位移偏离特征点位移平均值最大的特征点匹配对作为疑点匹配对，如果该疑点匹配对的特征点的偏离值大于预设的位移偏移值阈值时，则剔除该疑点匹配点。随后继续使用for循环计算位移均值并寻找偏离匹配对，直到所有特征点匹配对相对于位移平均值的偏离值都小于预设的位移偏移值阈值。
最终采用经过细过滤的特征点匹配对M(V1”，V2_adjusted’)，对图像I1和图像I2进行位移和放大修正，最终准确的获取图像I1和图像I2中覆盖相同范围的嘴部图像，如图3所示。其中，图像I1对应的嘴部图像如图3中左边的图像所示，图像I2对应的嘴部图像如图3中右边的图像所示。
应理解，上述确定相同范围和相同中心位置的嘴部图像的方法仅为示例性说明，并不对本申请构成任何限定。本申请还可以通过其他方法确定相同范围和相同中心位置的嘴部图像。
在确定了第一目标图像与第二目标图像中的相同范围和相同中心位置的嘴部图像之后，下面对确定该第一目标用户的面部表情是否处于稳定状态的方法进行详细说明。
可选地，通过计算图像I1与图像I2的相似度，当该相似度大于或等于预设的第一阈值时，确定该第一目标用户的面部表情处于稳定状态。
具体而言，经过上述确定第一目标图像与第二目标图像中的相同范围和相同中心位置的嘴部图像的方法处理之后，图像I1与图像I2中已经包括了相同范围和相同中心位置的第一目标用户的嘴部图像，如图3所示。
在计算图像I1与图像I2的相似度时，首先计算图像I1中的每个像素值与图像I2中的对应的像素值的差值，并对该差值取绝对值。将所获得的所有像素值差值的绝对值与预设的像素值差值的阈值进行比较，记录像素值差值的绝对值大于该预设的像素值差值的阈值的像素点的数量，计算该像素点的数量占整个图像I1或图像I2的总的像素点数量的比例，该比例即为图像I1与图像I2的差异值。当该比例小于或等于预设的比例阈值时，
则代表图像I1与图像I2的相似度很高，进一步确定出图像I1与图像I2之间的相似度，且当该相似度大于或者等于预设的第一阈值时，表明第一目标用户的面部表情处于稳定状态。
应理解，上述仅以通过计算相似度确定用户的面部表情的方法为例进行说明，但本申请并不限于此。还可以通过人脸建模的方法(例如，通过确定图像I1与图像I2中面部运动单元的位置，并对比图像I1与图像I2中该运动单元的位移)确定用户的面部表情是否处于稳定状态。或者，还可以通过提取面部特征点，再通过对比图像I1与图像I2中的面部特征点的位移来判断用户的面部表情是否处于稳定期。本申请对此不作任何限定。
当确定第一目标用户的面部表情处于稳定状态之后，就可以进行下一步，以确定用户的面部表情是否包括笑脸。当手机设备检测到用户的面部表情包括笑脸时，则手机设备相机快门自动激活，为用户进行拍照。
在检测用户的面部表情是否包括笑脸时，可以通过局部二值模式(Local Binary Pattern，LBP)的特征提取方法或者其他图像处理中的特征提取方法对用户的整个面部图像或者部分面部图像(例如，用户的下半部分面部图像)进行特征提取，然后将提取的特征点送入已经训练好的分类器(例如，分类器可以为支持向量机(Support Vector Machine，SVM))，并判断该面部图像是否包括笑脸图像。此外，还可以通过其他笑容检测方法判断用户的面部图像是否包括笑脸图像，例如，可以将用户面部图像送入训练好的人工神经网络模型(例如，卷积神经网络(Convolutional Neural Network，CNN)中直接进行分类，进而确定用户的面部图像是否包括笑脸图像。本申请对此不作限定。
下面对本申请的另一种自拍的策略进行说明。
当检测到用户面部表情处于稳定状态，且此时用户的面部包括笑脸时，自动启动快门进行拍照。然而，对于笑容稳定期持续时间较长的用户而言，可能会出现连续拍摄多张照片的情况。
因此，当检测到用户的面部表情处于稳定状态时，增加判断条件，例如，对于当前自拍，判断该稳定状态是不是第一次出现，只有当该稳定状态为在当前自拍中首次出现时，才检测用户的面部表情是否包括笑脸。
在步骤S110中，该第一目标图像的获取时刻与该第二目标图像的获取时刻之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
具体而言，当该时间间隔设置得非常短时，(例如，该时间间隔设置为20毫秒)，由于该时间间隔设置得非常短，用户的面部表情在该20毫秒内可能只发生了很小的变化(例如，用户的嘴部关键点只发生了微小的位移)。此时，由于用户的面部表情只发生了很小的变化，可能会导致系统进行误判，即，系统有可能认为该微小变化是由系统噪声引起的变化(例如，系统将该微小变化误判为用户在拍照过程中由于手机设备发生了晃动而引起的变化)，并非是在该时间间隔内用户面部表情发生了变化，进而导致系统无法对用户的面部表情是否处于稳定状态进行准确地判断；
此外，该时间间隔也不能设置得过长，当该时间间隔设置得过长时(例如，该时间间隔设置为500毫秒)，由于该时间间隔设置得过长，有可能导致系统错过用户面部表情的笑容饱和期，导致系统无法在用户的笑容饱和期对用户进行拍照。
因此，本申请提供的技术方案，将该时间间隔设置在一个区间内，即该时间间隔小
于或等于预设的第二阈值，且大于或等于预设的第三阈值。从而保证系统即可以以足够高的频率分析用户的面部表情是否处于稳定状态，又不会因为该时间间隔设置过小导致系统进行误判，占用手机设备的计算资源；并且也不会因为该时间间隔设置得过长导致系统错过用户的笑容饱和期，从而保证即改善了用户的体验，又降低了系统能耗。
需要说明的是，该时间间隔的取值除了满足上述的处于预设区间以内之外，还应当满足为手机设备的摄像头的图像获取周期的整数倍这一条件，例如，该摄像头捕捉图像的刷新率为50帧每秒，即每隔20毫秒捕捉一张图像，则该摄像头的图像获取周期为20毫秒。因此，该时间间隔应当为20毫秒的整数倍(例如，该时间间隔为20毫秒、40毫秒、60毫秒等，此处不作一一列举)。此外，用户通过手机设备的摄像头进行自拍时，随着用户面部距离摄像头的远近发生变化时，该时间间隔的取值也会随之出现上下浮动(即，该时间间隔的取值会相应变大或者变小)。例如，当用户面部距离摄像头较近时，由于摄像头能够捕捉到用户面部的较多细节，此时相对的干扰(例如，图像噪声)也较小，则可以适当减小该时间间隔的取值；相反地，当用户面部距离摄像头较远时，由于摄像头能够捕捉到用户面部的较少细节，此时可以适当增大该时间间隔的取值。通过根据用户面部距离摄像头的远近实时且灵活地改变该时间间隔的取值，从而提高系统整体的准确度，并且降低功耗。
还需要说明的是，用于判断用户面部表情是否处于稳定状态的相似度对应的第一阈值的取值会随着该时间间隔的取值的变化而发生变化。当该时间间隔设置较短时，则在该时间间隔内用户的面部表情的变化幅度也较小，相应地在该时间间隔内用户的面部表情的相似度也就越高，因此，应该将相似度对应的第一阈值相应地也设置得较大，以避免由于该第一阈值设置得较小，导致对用户的面部表情是否处于稳定状态造成误判；
相反地，当该时间间隔设置较大时，则在该时间间隔内用户的面部表情的变化幅度也较大，相应地在该时间间隔内用户的面部表情的相似度也就越低，因此，应该将相似度对应的第一阈值相应地也设置得较小，以避免由于该第一阈值设置得较大，导致对用户的面部表情是否处于稳定状态造成误判。上述确定第一目标用户的面部表情是否处于稳定状态是针对单人自拍模式而言的。下面针对多人自拍模式，对如何在多人自拍模式中确定第一目标用户的方法进行说明。
针对多用户自拍模式，手机设备组需要从至少两个用户中确定该第一目标用户。
具体而言，当同时有多个人通过同一前置摄像头进行自拍时，可以从该多人中确定一人作为后续确定面部表情是否处于稳定状态的第一目标用户。例如，手机设备可以将距离摄像头最近的用户确定为该第一目标用户；或者将多人当中面部图像所占像素数最多的用户确定为该第一目标用户；或者根据用户操作，将与该用户操作对应的用户确定为该第一目标用户，例如，当手机的屏幕中同时出现多个人的实时显示画面时，由其中任意一个用户自行在实时显示的预览屏幕上点击画面中的某一用户的头像进行第一目标用户的选取；或者将与存储器中存储的图像对应的用户确定为该第一目标用户，例如当有多个用户同时出现在了同一显示屏幕中时，其中该多个用户中的某个用户的头像作为通讯录中该用户的头像，则手机设备可以根据通讯录中保存的该用户的头像，确定第一目标用户，即从该多个用户中将与该头像对应的用户确定为第一目标用户。
根据本申请的自拍的方法，通过首先确定用户的面部表情是否处于稳定状态，在确定用户的面部表情处于稳定状态的基础上，再确定此时用户的面部表情是否包括笑脸，
最终在确定用户的面部表情为笑容最灿烂的状态的前提下，再自动控制相机快门通过摄像头进行拍照，记录用户笑容最灿烂的图像。避免仅仅根据用户的面部表情是否包括笑脸来控制相机快门通过摄像头所导致的对无效照片的拍摄，即降低了无效照片拍摄的概率，并且改善了用户体验。
上文中，结合图1至图3，详细描述了根据本申请的自拍的方法，下面，将结合图4，详细描述根据本申请的自拍的装置。
图4示出了根据本申请的自拍的装置200的示意性框图。如图4所示，该装置200包括获取单元210、确定单元220和控制单元230。
获取单元210，用于获取第一目标图像和第二目标图像，该第一目标图像与该第二目标图像为在不同时刻获取的第一目标用户的面部图像；
确定单元220，用于根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态；
该确定单元220还用于：
当该第一目标用户的面部表情处于稳定状态时，确定该第一目标图像或该第二目标图像中是否包括笑脸；
控制单元230，用于当该第一目标图像或该第二目标图像中包括笑脸时，控制相机快门通过摄像头进行拍照。
可选地，该确定单元220具体用于：
计算该第一目标图像与该第二目标图像之间的相似度；
该确定单元220具体还用于：
当该相似度大于或等于预设的第一阈值时，确定该第一目标用户的面部表情处于稳定状态。
可选地，该确定单元220具体还用于：
当该第一目标用户的面部表情处于稳定状态，且该稳定状态为第一次出现时，确定该第一目标图像或该第二目标图像中是否包括笑脸。
可选地，该第一目标图像的获取时刻与该第二目标图像的获取时刻之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
可选地，该确定单元220还用于：
针对多用户自拍模式，从至少两个用户中确定该第一目标用户。
可选地，该确定单元220具体用于：
将距离摄像头最近的用户确定为该第一目标用户；或
将面部图像所占像素数最多的用户确定为该第一目标用户；或
根据用户操作，将与该用户操作对应的用户确定为该第一目标用户；或
将与存储器中存储的图像对应的用户确定为该第一目标用户。
可选地，该第一目标图像与该第二目标图像为覆盖该第一目标用户的相同面部范围的嘴部图像。
根据本申请的用于自拍的装置200可对应于本申请的用于自拍的方法100的实施主体，并且，该用于自拍的装置200中的各单元和上述其他操作和/或功能分别为了实现图1中的方法100的相应流程，为了简洁，在此不再赘述。
根据本申请的用于自拍的装置，通过首先确定用户的面部表情是否处于稳定状态，
在确定用户的面部表情处于稳定状态的基础上，再确定此时用户的面部表情是否包括笑脸，最终在确定用户的面部表情为笑容最灿烂的状态的前提下，再自动控制相机快门通过摄像头进行拍照，记录用户笑容最灿烂的图像。避免仅仅根据用户的面部表情是否包括笑脸来控制相机快门通过摄像头所导致的对无效照片的拍摄，即降低了无效照片拍摄的概率，并且改善了用户体验。并且根据用户表情稳定状态的判断结果，确定是否进行笑脸检测，当用户表情不处于稳定状态时，则不需要再进行复杂的笑脸检测，从而大幅降低系统能耗。
上文中，结合图1至图4，详细描述了根据本申请的自拍的方法，下面，将结合图5，详细描述根据本申请的终端设备。
图5示出了根据本申请的终端设备300的示意性框图。如图5所示，该终端设备300包括处理器310、存储器320和摄像头330。其中，存储器320用于存储指令，处理器310用于执行存储器320存储的指令，以控制相机快门通过摄像头330进行拍照。
存储器320可以包括易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储器也可以包括非易失性存储器(non-volatile memory)，例如快闪存储器(flash memory)、硬盘(hard disk drive，HDD)或固态硬盘(solid-state drive，SSD)；存储器320还可以包括上述种类的存储器的组合。
处理器310可以是中央处理器(central processing unit，CPU)、网络处理器(network processor，NP)或者CPU和NP的组合。处理器310还可以进一步包括硬件芯片。上述硬件芯片可以是专用集成电路(application-specific integrated circuit，ASIC)、可编程逻辑器件(programmable logic device，PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(complex programmable logic device，CPLD)、现场可编程逻辑门阵列(field-programmable gate array，FPGA)、通用阵列逻辑(generic array logic，GAL)或其任意组合。
该处理器310用于获取第一目标图像和第二目标图像，该第一目标图像与该第二目标图像为在不同时刻获取的第一目标用户的面部图像；
该处理器310还用于根据该第一目标图像和该第二目标图像，确定该第一目标用户的面部表情是否处于稳定状态；
该处理器310还用于当该第一目标用户的面部表情处于稳定状态时，确定该第一目标图像或该第二目标图像中是否包括笑脸；
该处理器310，还用于当该第一目标图像或该第二目标图像中包括笑脸时，控制相机快门通过该摄像头330进行拍照。
可选地，该处理器310具体用于：
计算该第一目标图像与该第二目标图像之间的相似度；
该处理器310具体还用于：
当该相似度大于或等于预设的第一阈值时，确定该第一目标用户的面部表情处于稳定状态。
可选地，该处理器310具体用于：
当该第一目标用户的面部表情处于稳定状态，且该稳定状态为第一次出现时，确定该第一目标图像或该第二目标图像中是否包括笑脸。
可选地，该第一目标图像的获取时刻与该第二目标图像的获取时刻之间的时间间隔
小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
可选地，该处理器310还用于：
针对多用户自拍模式，从至少两个用户中确定该第一目标用户。
可选地，该处理器310具体用于：
将距离摄像头最近的用户确定为该第一目标用户；或
将面部图像所占像素数最多的用户确定为该第一目标用户；或
根据用户操作，将与该用户操作对应的用户确定为该第一目标用户；或
将与存储器中存储的图像对应的用户确定为该第一目标用户。
可选地，该第一目标图像与该第二目标图像为覆盖该第一目标用户的相同面部范围的嘴部图像。
根据本申请的终端设备300可对应于本申请的用于自拍的方法100的实施主体，并且，该终端设备300中的各单元和上述其他操作和/或功能分别为了实现图1中的方法100的相应流程，为了简洁，在此不再赘述。
根据本申请的用于自拍的终端设备，通过首先确定用户的面部表情是否处于稳定状态，在确定用户的面部表情处于稳定状态的基础上，再确定此时用户的面部表情是否包括笑脸，最终在确定用户的面部表情为笑容最灿烂的状态的前提下，再自动控制相机快门通过摄像头进行拍照，记录用户笑容最灿烂的图像。避免仅仅根据用户的面部表情是否包括笑脸来控制相机快门通过摄像头所导致的对无效照片的拍摄，即降低了无效照片拍摄的概率，并且改善了用户体验。并且根据用户表情稳定状态的判断结果，确定是否进行笑脸检测，当用户表情不处于稳定状态时，则不需要再进行复杂的笑脸检测，从而大幅降低系统能耗。
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的
目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求所述的保护范围为准。
Claims (21)
- 一种自拍的方法，其特征在于，所述方法包括：获取第一目标图像和第二目标图像，所述第一目标图像与所述第二目标图像为在不同时刻获取的第一目标用户的面部图像；根据所述第一目标图像和所述第二目标图像，确定所述第一目标用户的面部表情是否处于稳定状态；当所述第一目标用户的面部表情处于稳定状态时，确定所述第一目标图像或所述第二目标图像中是否包括笑脸；当所述第一目标图像或所述第二目标图像中包括笑脸时，控制相机快门通过摄像头进行拍照。
- 根据权利要求1所述的方法，其特征在于，所述根据所述第一目标图像和所述第二目标图像，确定所述第一目标用户的面部表情是否处于稳定状态，包括：计算所述第一目标图像与所述第二目标图像之间的相似度；当所述相似度大于或等于预设的第一阈值时，确定所述第一目标用户的面部表情处于稳定状态。
- 根据权利要求1或2所述的方法，其特征在于，所述确定所述第一目标图像或所述第二目标图像中是否包括笑脸，包括：当所述第一目标用户的面部表情处于稳定状态，且所述稳定状态为第一次出现时，确定所述第一目标图像或所述第二目标图像中是否包括笑脸。
- 根据权利要求1至3中任一项所述的方法，其特征在于，所述第一目标图像的获取时刻与所述第二目标图像的获取时刻之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
- 根据权利要求1至4中任一项所述的方法，其特征在于，所述方法还包括：针对多用户自拍模式，从至少两个用户中确定所述第一目标用户。
- 根据权利要求5所述的方法，其特征在于，所述从至少两个用户中确定所述第一目标用户，包括：将距离摄像头最近的用户确定为所述第一目标用户；或将面部图像所占像素数最多的用户确定为所述第一目标用户；或根据用户操作，将与所述用户操作对应的用户确定为所述第一目标用户；或将与存储器中存储的图像对应的用户确定为所述第一目标用户。
- 根据权利要求1至6中任一项所述的方法，其特征在于，所述第一目标图像与所述第二目标图像为覆盖所述第一目标用户的相同面部范围的嘴部图像。
- 一种自拍的装置，其特征在于，所述装置包括：获取单元，用于获取第一目标图像和第二目标图像，所述第一目标图像与所述第二目标图像为在不同时刻获取的第一目标用户的面部图像；确定单元，用于根据所述第一目标图像和所述第二目标图像，确定所述第一目标用户的面部表情是否处于稳定状态；所述确定单元还用于：当所述第一目标用户的面部表情处于稳定状态时，确定所述第一目标图像或所述第二目标图像中是否包括笑脸；控制单元，用于当所述第一目标图像或所述第二目标图像中包括笑脸时，控制相机快门通过摄像头进行拍照。
- 根据权利要求8所述的装置，其特征在于，所述确定单元具体用于：计算所述第一目标图像与所述第二目标图像之间的相似度；所述确定单元具体还用于：当所述相似度大于或等于预设的第一阈值时，确定所述第一目标用户的面部表情处于稳定状态。
- 根据权利要求8或9所述的装置，其特征在于，所述确定单元具体还用于：当所述第一目标用户的面部表情处于稳定状态，且所述稳定状态为第一次出现时，确定所述第一目标图像或所述第二目标图像中是否包括笑脸。
- 根据权利要求8至10中任一项所述的装置，其特征在于，所述第一目标图像的获取时刻与所述第二目标图像的获取时刻之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
- 根据权利要求8至11中任一项所述的装置，其特征在于，所述确定单元还用于：针对多用户自拍模式，从至少两个用户中确定所述第一目标用户。
- 根据权利要求12所述的装置，其特征在于，所述确定单元具体用于：将距离摄像头最近的用户确定为所述第一目标用户；或将面部图像所占像素数最多的用户确定为所述第一目标用户；或根据用户操作，将与所述用户操作对应的用户确定为所述第一目标用户；或将与存储器中存储的图像对应的用户确定为所述第一目标用户。
- 根据权利要求8至13中任一项所述的装置，其特征在于，所述第一目标图像与所述第二目标图像为覆盖所述第一目标用户的相同面部范围的嘴部图像。
- 一种终端设备，其特征在于，包括：存储器，处理器和摄像头；所述存储器用于存储指令；所述处理器用于调用所述存储器中的指令执行以下步骤：所述处理器，用于获取第一目标图像和第二目标图像，所述第一目标图像与所述第二目标图像为在不同时刻获取的第一目标用户的面部图像；所述处理器，用于根据所述第一目标图像和所述第二目标图像，确定所述第一目标用户的面部表情是否处于稳定状态；所述处理器，还用于当所述第一目标用户的面部表情处于稳定状态时，确定所述第一目标图像或所述第二目标图像中是否包括笑脸；所述处理器，还用于当所述第一目标图像或所述第二目标图像中包括笑脸时，控制相机快门通过所述摄像头进行拍照。
- 根据权利要求15所述的终端设备，其特征在于，所述处理器具体用于：计算所述第一目标图像与所述第二目标图像之间的相似度；所述处理器具体还用于：当所述相似度大于或等于预设的第一阈值时，确定所述第一目标用户的面部表情处于稳定状态。
- 根据权利要求15或16所述的终端设备，其特征在于，所述处理器具体用于：当所述第一目标用户的面部表情处于稳定状态，且所述稳定状态为第一次出现时， 确定所述第一目标图像或所述第二目标图像中是否包括笑脸。
- 根据权利要求15至17中任一项所述的终端设备，其特征在于，所述第一目标图像的获取时刻与所述第二目标图像的获取时刻之间的时间间隔小于或等于预设的第二阈值，且大于或等于预设的第三阈值。
- 根据权利要求15至18中任一项所述的终端设备，其特征在于，所述处理器还用于：针对多用户自拍模式，从至少两个用户中确定所述第一目标用户。
- 根据权利要求19所述的终端设备，其特征在于，所述处理器具体用于：将距离摄像头最近的用户确定为所述第一目标用户；或将面部图像所占像素数最多的用户确定为所述第一目标用户；或根据用户操作，将与所述用户操作对应的用户确定为所述第一目标用户；或将与存储器中存储的图像对应的用户确定为所述第一目标用户。
- 根据权利要求15至20中任一项所述的终端设备，其特征在于，所述第一目标图像与所述第二目标图像为覆盖所述第一目标用户的相同面部范围的嘴部图像。
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201780085116.1A CN110268702A (zh) | 2017-04-13 | 2017-04-13 | 自拍的方法、装置和终端设备 |
| PCT/CN2017/080335 WO2018188007A1 (zh) | 2017-04-13 | 2017-04-13 | 自拍的方法、装置和终端设备 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2017/080335 WO2018188007A1 (zh) | 2017-04-13 | 2017-04-13 | 自拍的方法、装置和终端设备 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2018188007A1 true WO2018188007A1 (zh) | 2018-10-18 |
Family
ID=63792283
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2017/080335 WO2018188007A1 (zh) | 2017-04-13 | 2017-04-13 | 自拍的方法、装置和终端设备 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110268702A (zh) |
| WO (1) | WO2018188007A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112712006A (zh) * | 2020-12-25 | 2021-04-27 | 四川云从天府人工智能科技有限公司 | 目标图片抓拍方法、系统、介质及装置 |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101355652A (zh) * | 2007-07-27 | 2009-01-28 | 奥林巴斯映像株式会社 | 摄像装置和摄像装置的控制方法 |
| CN101625505A (zh) * | 2008-07-10 | 2010-01-13 | 华晶科技股份有限公司 | 脸部辨识的连续拍照方法 |
| CN101646018A (zh) * | 2008-08-08 | 2010-02-10 | 佛山普立华科技有限公司 | 摄影装置及其自拍方法 |
| CN102710865A (zh) * | 2012-06-18 | 2012-10-03 | 宇龙计算机通信科技(深圳)有限公司 | 具有拍照功能的移动终端及其人像自拍方法 |
| CN103024275A (zh) * | 2012-12-17 | 2013-04-03 | 东莞宇龙通信科技有限公司 | 自动拍摄方法及终端 |
| WO2016188304A1 (zh) * | 2016-03-04 | 2016-12-01 | 中兴通讯股份有限公司 | 拍照的方法及装置 |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4453721B2 (ja) * | 2007-06-13 | 2010-04-21 | ソニー株式会社 | 画像撮影装置及び画像撮影方法、並びにコンピュータ・プログラム |
| JP4356778B2 (ja) * | 2007-06-25 | 2009-11-04 | ソニー株式会社 | 画像撮影装置及び画像撮影方法、並びにコンピュータ・プログラム |
| US8750578B2 (en) * | 2008-01-29 | 2014-06-10 | DigitalOptics Corporation Europe Limited | Detecting facial expressions in digital images |
-
2017
- 2017-04-13 CN CN201780085116.1A patent/CN110268702A/zh active Pending
- 2017-04-13 WO PCT/CN2017/080335 patent/WO2018188007A1/zh active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101355652A (zh) * | 2007-07-27 | 2009-01-28 | 奥林巴斯映像株式会社 | 摄像装置和摄像装置的控制方法 |
| CN101625505A (zh) * | 2008-07-10 | 2010-01-13 | 华晶科技股份有限公司 | 脸部辨识的连续拍照方法 |
| CN101646018A (zh) * | 2008-08-08 | 2010-02-10 | 佛山普立华科技有限公司 | 摄影装置及其自拍方法 |
| CN102710865A (zh) * | 2012-06-18 | 2012-10-03 | 宇龙计算机通信科技(深圳)有限公司 | 具有拍照功能的移动终端及其人像自拍方法 |
| CN103024275A (zh) * | 2012-12-17 | 2013-04-03 | 东莞宇龙通信科技有限公司 | 自动拍摄方法及终端 |
| WO2016188304A1 (zh) * | 2016-03-04 | 2016-12-01 | 中兴通讯股份有限公司 | 拍照的方法及装置 |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112712006A (zh) * | 2020-12-25 | 2021-04-27 | 四川云从天府人工智能科技有限公司 | 目标图片抓拍方法、系统、介质及装置 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110268702A (zh) | 2019-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6961797B2 (ja) | プレビュー写真をぼかすための方法および装置ならびにストレージ媒体 | |
| US20200302154A1 (en) | Image processing method, apparatus, storage medium, and electronic device | |
| US8780221B2 (en) | Facial expression recognition apparatus, image sensing apparatus, facial expression recognition method, and computer-readable storage medium | |
| US8391645B2 (en) | Detecting orientation of digital images using face detection information | |
| CN105072327B (zh) | 一种防闭眼的人像拍摄处理的方法和装置 | |
| WO2018103314A1 (zh) | 一种拍照方法、装置、终端及存储介质 | |
| WO2019071613A1 (zh) | 一种图像处理方法及装置 | |
| US20170161553A1 (en) | Method and electronic device for capturing photo | |
| CN108229369A (zh) | 图像拍摄方法、装置、存储介质及电子设备 | |
| EP3644599B1 (en) | Video processing method and apparatus, electronic device, and storage medium | |
| CN108830892B (zh) | 人脸图像处理方法、装置、电子设备及计算机可读存储介质 | |
| CN103685940A (zh) | 一种通过表情识别拍摄照片的方法 | |
| TW201445454A (zh) | 提升人臉辨識率的影像處理系統及影像處理方法 | |
| WO2016176990A1 (zh) | 移动终端基于高质量眼纹图像进行解锁的方法及系统 | |
| US11006122B2 (en) | Static video recognition | |
| CN108200337B (zh) | 拍照处理的方法、装置、终端及存储介质 | |
| WO2019214574A1 (zh) | 图像拍摄方法、装置及电子终端 | |
| JP2016081249A (ja) | 情報処理装置および情報処理方法 | |
| WO2016165107A1 (zh) | 一种采集指纹的方法、指纹采集器以及终端 | |
| WO2021008205A1 (zh) | 图像处理 | |
| CN108605087A (zh) | 终端的拍照方法、拍照装置和终端 | |
| CN111654624A (zh) | 拍摄提示方法、装置及电子设备 | |
| CN103873755B (zh) | 跳跃人像拍摄系统及方法 | |
| CN115862081A (zh) | 图像采集方法、装置、设备和计算机可读存储介质 | |
| WO2018188007A1 (zh) | 自拍的方法、装置和终端设备 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17905660 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 17905660 Country of ref document: EP Kind code of ref document: A1 |