WO2020103526A1

WO2020103526A1 - Photographing method and device, storage medium and terminal device

Info

Publication number: WO2020103526A1
Application number: PCT/CN2019/104549
Authority: WO
Inventors: 侯允; 刘耀勇; 陈岩
Original assignee: Oppo广东移动通信有限公司
Priority date: 2018-11-21
Filing date: 2019-09-05
Publication date: 2020-05-28
Also published as: CN109348135A

Abstract

Provided in the embodiments of the present application is a photographing method, comprising: when photographing, acquiring a preview image of a user to be photographed within a photographing area; recognizing the expression comprised in the preview image according to a preset expression recognition model; when detected that the user is in a stationary state and the expression is a target expression, triggering photographing so as to acquire an image of the user, which thus does not require the user to manually trigger or rely on a photographing assistance tool, and allows the user to automatically take a picture, which may improve the convenience of taking pictures.

Description

Photographing method, device, storage medium and terminal equipment

This application requires the priority of the Chinese patent application submitted to the China Patent Office on November 21, 2018, with the application number 201811392429.4 and the application name "photographing method, device, storage medium and terminal equipment", the entire content of which is incorporated by reference in In this application.

Technical field

Embodiments of the present application relate to the technical field of terminal equipment, and in particular, to a photographing method, device, storage medium, and terminal equipment.

Background technique

The terminal device is generally equipped with a camera, and with the development of the camera technology of the terminal device, most terminal devices will be equipped with at least two cameras, and the camera function of the terminal device is becoming more and more abundant.

Summary of the invention

A photographing method, device, storage medium, and terminal device provided in the embodiments of the present application can improve the convenience of photographing.

In a first aspect, an embodiment of the present application provides a photographing method, including:

When shooting in the preset shooting mode, obtain a preview image of the user who is shot in the shooting area;

Identify the expression included in the preview image according to a preset expression recognition model;

When it is detected that the user is in a stationary state, and the user's expression is a target expression, a photograph is triggered to collect an image of the user.

In a second aspect, an embodiment of the present application provides a photographing device, including:

The image acquisition module is used to acquire the preview image of the user who is shot in the shooting area when shooting in the preset shooting mode;

An expression recognition module, configured to recognize the expression included in the preview image according to a preset expression recognition model;

The photographing triggering module is used to trigger the photographing to collect the user's image when it is detected that the user is at rest and the user's expression is the target expression.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, which is executed by a processor:

According to a fourth aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes:

BRIEF DESCRIPTION

FIG. 1 is a schematic flowchart of a photographing method provided by an embodiment of the present application;

2 is a schematic flowchart of another photographing method provided by an embodiment of the present application;

3 is a schematic flowchart of another photographing method provided by an embodiment of the present application;

4 is a schematic flowchart of another photographing method provided by an embodiment of the present application;

5 is a schematic flowchart of another photographing method provided by an embodiment of the present application;

6 is a structural block diagram of a camera device provided by an embodiment of the present application;

7 is a schematic structural diagram of a terminal device according to an embodiment of this application;

FIG. 8 is a schematic structural diagram of another terminal device according to an embodiment of the present application.

detailed description

The technical solution of the present application will be further described below with reference to the drawings and through specific implementations. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limit the present application. In addition, it should be noted that, for ease of description, the drawings only show parts, but not all structures related to the present application.

Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowchart describes the processes as sequential processing, many of the processes can be implemented in parallel, concurrently, or simultaneously. In addition, the sequence of the processes can be rearranged. The processing may be terminated when its operation is completed, but may also have an additional flow not included in the drawings. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and so on.

An embodiment of the present application provides a photographing method, including:

In an embodiment, the acquiring the preview image of the user captured in the shooting area may include:

Acquire the first number of preview images of the users photographed in the shooting area within a preset time;

The recognizing the expression included in the preview image according to the preset expression recognition model includes:

Identifying the expression included in the first number of preview images through a preset expression recognition model;

If the first number of preview images all include a target expression, it is determined that the user's expression is the target expression.

In one embodiment, after acquiring the first number of preview images of the users photographed in the shooting area, the method further includes:

Detecting changes in pixels in the first number of preview images;

If the change of the pixel points meets the preset condition, it is determined that the user is in a stationary state.

In one embodiment, the triggering to take a picture to collect the image of the user includes:

Trigger a photo to collect a second number of images of the user.

In one embodiment, when it is detected that the user is in a stationary state, the method further includes:

Prompt for shooting;

After triggering to take a picture to collect the image of the user, the method further includes:

Prompt for completion.

In one embodiment, before the triggering to take a picture to collect the image of the user, the method further includes:

Determine the profile information and face angle of the user in the preview image;

Determine the user's photographing posture according to the profile information and the face angle;

When it is detected that the user is in a stationary state, and the user's expression is a target expression, triggering a photograph to collect an image of the user includes:

When it is detected that the user is in a stationary state, the expression of the user is a target expression, and the photographing posture is a preset posture, a photograph is triggered to collect images of the user.

In an embodiment, before recognizing the expression included in the preview image according to a preset expression recognition model, the method further includes:

Input the preset face data set into the preset model for training to obtain the preset expression recognition model;

Wherein, the preset face data set includes preset face images, face key point data and expressions, and the preset expression recognition model is used to identify face key point data in the input image, and according to the The face key point data determines the expression included in the input image.

FIG. 1 is a schematic flowchart of a photographing method provided by an embodiment of the present application. The method may be performed by a photographing device, where the device may be implemented by software and / or hardware, and may be generally integrated in a terminal device or other In devices with an operating system. As shown in Figure 1, the method includes:

S110. When shooting in the preset photographing mode, obtain a preview image of the user who is shot in the shooting area.

The user can take a selfie through the camera of the terminal device. When taking a picture through the handheld terminal device, the user needs to manually trigger the camera button, which is very inconvenient in the self-timer mode. In addition, users can use the selfie stick to trigger the camera button, but this method needs to rely on shooting aids, which is still not convenient.

Wherein, the preset photographing mode may be a mode in which a preset camera in the terminal device is turned on to take a picture, or a mode in which a camera in the terminal device may be taken according to preset shooting parameters, and the terminal device may be a smartphone , Tablet or other electronic devices with corresponding operating systems. Exemplarily, the preset photographing mode may be a self-timer mode, that is, a mode for taking pictures through a front camera of a terminal device.

When shooting the user through the camera, the captured user will appear in the shooting area. The preview image turns on the camera of the terminal device, and the camera is aimed at the area to be captured. The camera can be captured on the display screen of the terminal device. Real-time image of the area, which will include the image of the user being photographed. The preview image only displays the real-time image obtained by the camera, but does not record the real-time image.

The preview image of the user in the shooting area can reflect the current user's status in real time, and the user's status can be identified by acquiring the preview image of the user shot in the shooting area. You can cache the current preview screen and perform image processing on the preview screen to identify the user form in the preview image. Generally, the preview image will not be stored, and the preview image will be stored only when the photo is triggered.

S111. Identify the expression included in the preview image according to a preset expression recognition model.

The preset expression recognition model may be a machine learning model that has been trained, and the expression recognition model may recognize the expression included in the input image. Input the buffered preview image of the current shooting area into the expression recognition model to obtain the expression included in the preview expression.

Optionally, before recognizing the expression included in the preview image according to a preset expression recognition model, an operation of training an expression recognition model may be further included, which may be: inputting a preset face data set into the preset model for training, to Obtain a preset expression recognition model; wherein, the preset face data set includes preset face images, face key point data, and expressions, and the preset expression recognition model is used to identify people in the input image Face key point data, and determine the expression included in the input image according to the face key point data.

Wherein, the preset model may be a machine learning model, the preset face data set is input into the preset model to train the preset model, and the preset model establishes a mapping relationship between the face image, face key point data and expression, To generate an expression recognition model.

The preset face data set includes multiple face images, covering face images of different poses, different angles, different expressions and whether to wear accessories, and each face image is marked with corresponding face key point data and expressions. The face key point data is key point data for distinguishing expressions. Illustratively, the face key point data includes the angle of the corner of the mouth. If the angle of the corner of the mouth of a face image is upward, it can be determined that the image may contain a smiling expression. One face image can correspond to multiple face key point data, and the facial expression can be determined according to multiple face key data.

The facial expression recognition model can first recognize the key point information of the face in the preview image, and then can determine the corresponding facial expression based on the recognized facial key point information as the facial expression included in the preview image.

S112. When it is detected that the user is in a stationary state, and the user's expression is a target expression, trigger a photograph to collect an image of the user.

Among them, when acquiring the preview image of the user photographed in the shooting area, it can be detected whether the user moves according to the preview image, when it is detected that the user has not moved, that is, when the user is in a stationary state, and recognized by a preset expression When the model recognizes that the expression included in the preview image is the target expression, it may trigger a photograph to collect the user's image.

Among them, when detecting whether the user has moved, in addition to the user's own movement, it also includes the shaking of the user's preview image due to the movement of the terminal device. The movement of the two cases will make the collected user image unclear , So you need to trigger a photo when the user is detected to be still, you can collect a clearer image.

When taking photos in the preset photo mode, the user will take a suitable expression to take photos. For example, if the preset photo mode is a self-timer model, the user will generally take photos with a smiling expression. The smiling expression can be used as the target expression, that is, when the user's expression is detected as a smiling expression, a photograph is triggered to collect an image containing the user's smiling expression.

Triggered photographing may be by sending a trigger instruction to a functional component for managing photographing, so that the functional component executes a photographing process, collects the user's image through a camera, and stores the collected user's image in the storage space of the terminal device in.

In the embodiment of the present application, the user's image is collected by triggering a photograph when it is detected that the user is in a stationary state and the user's expression is the target expression. When the target expression is detected and the user does not move, the camera is triggered. Not only can automatic photography be achieved, but also images containing suitable expressions can be captured, and the image will not be blurred due to user movement, which can improve the clarity of the image degree.

In a photographing method provided in an embodiment of the present application, when shooting in a preset photographing mode, a preview image of a user who is photographed in a shooting area is acquired; and the preview image is identified according to a preset expression recognition model Expression; when it is detected that the user is in a stationary state, and the user's expression is the target expression, a photo is triggered to collect an image of the user. By adopting the above technical solution, automatic photographing can be performed according to the expression of the user when the user is in a stationary state, and the automatic photographing can be achieved without the user manually triggering or relying on the photographing auxiliary tool, and the convenience of photographing can be improved.

FIG. 2 is a schematic flowchart of another photographing method provided by an embodiment of the present application. On the basis of the technical solution provided by the above embodiment, the operation of obtaining a preview image of a user photographed in the shooting area is optimized, and Optionally, as shown in FIG. 2, the method includes:

S120. When shooting in a preset photographing mode, within a preset time, obtain a first number of preview images of users who are shot in the shooting area.

The preset time may be a time period preset by the system. Exemplarily, the preset time may be 2 seconds or 3 seconds, and the first number may be a preset value, which may be 5, 7, or 10. Within the preset time, the first number of preview images of the captured users in the shooting area is acquired, that is, within the preset time period of the system, several preview images of the captured users are continuously acquired, and each preview image is one Frame preview image.

S121. Identify the expression included in the first number of preview images through a preset expression recognition model.

S122. If the first number of preview images all include a target expression, determine that the user's expression is the target expression.

The collected first number of preview images are input into a preset expression recognition model for recognition, and the expression recognition model separately recognizes the expression included in each preview image.

When users take pictures, they generally need to make a suitable expression and then take pictures. In the early stage, users will adjust their expressions. Therefore, in this embodiment of the present application, it is possible to determine whether the user has done a good job by acquiring the first number of preview images of the users photographed in the shooting area within a preset time, and separately identifying the user's expression in each preview image Emoji for taking pictures. If the first number of preview images all include the target expression, it can be determined that the user has made a photo expression.

S123. When it is detected that the user is in a stationary state, and the user's expression is the target expression, trigger a photograph to collect an image of the user.

For a specific implementation manner, reference may be made to the related description above, and details are not described herein again.

Optionally, as shown in FIG. 3, after acquiring the first number of preview images of the users photographed in the shooting area within a preset time, the following operations are further included:

S124. Detect a change in pixels in the first number of preview images.

S125. If the change of the pixel points meets a preset condition, determine that the user is in a stationary state.

Wherein, the pixels in the preview image of each frame can be detected separately, and the change of the pixels can be determined according to the difference between the pixels in the preview image of each frame and the pixels in the preview image of the adjacent frame.

The preset condition may be a judgment condition for determining that the degree of change of a pixel is less than a preset degree. Exemplarily, the preset condition may be a ratio of pixels whose pixel value changes less than a preset value are less than a preset proportion. If the number of the preview images is greater than two, the change of pixels in the preview images of any two adjacent frames can be determined separately, if the change of pixels in the preview images of any two adjacent frames conforms to the preset Conditions, it can be determined that the user is in a stationary state.

When a user takes a picture, the user will adjust their own expression and posture in addition to their own expressions. Therefore, it is necessary to determine whether the user is in a stationary state. When it is determined that the user is in a stationary state, an automatic photograph is triggered to avoid collecting a blurred image.

In this embodiment of the present application, by obtaining a first number of preview images and separately identifying the expressions included in each frame of the preview image, an automatic photograph can be triggered when it is determined that the user has maintained a suitable target expression, and a suitable target expression can be collected Images to improve the accuracy of automatic photos.

FIG. 4 is a schematic flowchart of another photographing method provided by an embodiment of the present application. On the basis of the technical solution provided by any of the foregoing embodiments, the operation of triggering a photograph to collect the user ’s image is optimized, optional Ground, as shown in FIG. 4, the method includes:

S130. When shooting in the preset photographing mode, obtain a preview image of the user who is shot in the shooting area.

S131. Identify the expression included in the preview image according to a preset expression recognition model.

For the specific implementation of the above operation, reference may be made to the related description above, and no further description is provided here.

S132. When it is detected that the user is in a stationary state and the user's expression is a target expression, trigger a photograph to collect a second number of images of the user.

Among them, since the embodiment of the present application automatically triggers the photographing, the user may not be satisfied with the image collected by the automatic photographing, and may choose to delete the image and perform the photographing again. Further, a second number of user images can be collected by triggering a photograph. The second number can be a system preset or a user setting. Collecting the second number of images can provide multiple options for the user, and the user can select a suitable one from the multiple images Keep to avoid the need to take pictures again.

Optionally, when the user is detected to be in a stationary state, the method further includes the following operations: performing a photographing prompt; and after triggering the photographing to collect the image of the user, further including: performing a prompt to complete.

Wherein, the shooting prompt and the completion prompt may be a sound prompt or a flash prompt through the terminal device. The shooting prompt is used to remind the user to take a picture soon. The user needs to keep the gesture until the picture is taken. The completion prompt is used to remind the user that the picture is taken, and the user can stop taking a picture.

In this embodiment of the present application, by triggering a photograph to collect a second number of images of the user, multiple captured images can be provided to the user for selection, and the user can select a satisfactory image from which to retain, reducing the probability of the user needing to re-photograph, and improving automatic The effectiveness of taking pictures.

FIG. 5 is a schematic flowchart of another photographing method provided by an embodiment of the present application. Based on the technical solution provided by any of the foregoing embodiments, optionally, as shown in FIG. 5, the method includes:

S140. When shooting in the preset shooting mode, obtain a preview image of the user who is shot in the shooting area.

S141. Identify the expression included in the preview image according to a preset expression recognition model.

S142. Determine the profile information and face angle of the user in the preview image.

Wherein, the profile information of the user is the boundary information between the user and the background in the shooting area, and the angle of the user's face is the relative angle between the plane where the user's facial features are located and the reference plane. The reference plane may be a plane perpendicular to the shooting angle. The user's facial features are three-dimensional in actual space, but a preset plane can be fitted according to the positions of the facial features, so as to minimize the distance between the actual position of the facial features and the preset plane, and the preset plane can be determined as the user's The plane where the five senses are.

The user's basic posture can be determined based on the user's contour information. Because the user's head is close to the shape of a sphere, the actual angle of the user's head may not be accurately determined based on the user's contour information, so it is necessary to determine the user's face angle Determine the posture of the user's head, and then determine the user's accurate photographing posture based on the profile information and the face angle.

S143: Determine the user's photographing posture according to the profile information and the face angle.

The purpose of the photos taken by users is different, and different shooting poses will be adopted. Illustratively, if the user is taking a more casual selfie, the photographing posture may be a more relaxed and casual posture, such as a posture that tilts the body or gestures the hands of scissors. If the user needs to take a photo of the ID, the user will take a photo in a correct manner. Determining the user's photographing posture according to the user's profile information and face angle may be: determining the user's body posture based on the user's profile information, and then determining the user's head posture based on the face angle, and then according to the body posture and head The posture of the department determines the posture of taking pictures.

The posture of the body and the posture of the head may be determined according to the preset posture recognition model, and the photographing posture may be determined according to the posture of the body and the posture of the head. The posture recognition model may be a trained machine learning model.

S144. When it is detected that the user is in a stationary state, the user's expression is a target expression, and the photographing posture is a preset posture, triggering a photograph to collect an image of the user.

Among them, the target expression and the preset gesture have a corresponding relationship with the preset photographing mode, that is, different preset photographing modes correspond to different target expressions and different preset gestures. When shooting in the preset photographing mode, the corresponding target facial expression and the corresponding preset posture are determined, so that automatic photographing can be triggered according to the user's posture and facial expression.

The user adopts different photographing modes, and the purpose of the photo that the user wants to obtain will be different. Exemplarily, if the user is in the self-portrait mode, the user generally wants to obtain a relaxed selfie, and if the user is in the photo-taking mode, the user generally wants to obtain a decent portrait. Therefore, in different preset photo modes, the user will make different postures and expressions, and correspondingly determine the user's expressions and postures to trigger the automatic photographing. The user can make the corresponding postures and expressions to trigger the automatic photographing. To collect photos containing the pose and expression.

In the embodiment of the present application, the automatic photographing is triggered by judging that the user is in a stationary state, the user's expression is the target expression, and the user's photographing posture is the preset posture. Appropriate photos can further improve the intelligence and accuracy of automatic photography.

An embodiment of the present application also provides a photographing device, including:

In some embodiments, the image acquisition module is also used to:

The expression recognition module is specifically used for:

In some embodiments, the image acquisition module is also used to:

Within a preset time, after acquiring a first number of preview images of the users photographed in the shooting area, detecting a change in pixels in the first number of preview images;

In some embodiments, the image acquisition module is also used to:

Trigger a photo to collect a second number of images of the user.

In some embodiments, the device further includes:

A prompting module for starting a shooting prompt when the user is detected to be in a stationary state;

The completion prompting module is used for prompting completion after triggering the photographing to collect the image of the user.

In some embodiments, it also includes:

The information determining module is used to determine the profile information and face angle of the user in the preview image before the triggering of the photograph to collect the image of the user;

A posture determination module, configured to determine the user's photo posture based on the profile information and the face angle;

The photo trigger module is specifically used for:

In some embodiments, it also includes:

The model training module is used to input the preset face data set into the preset model for training before identifying the expression included in the preview image according to the preset expression recognition model to obtain the preset expression recognition model ;

FIG. 6 is a structural block diagram of a photographing device according to an embodiment of the present application. The device can perform a photographing method. As shown in FIG. 6, the device includes:

The image acquisition module 201 is used to acquire the preview image of the user who is photographed in the shooting area when shooting in the preset photographing mode;

The expression recognition module 202 is configured to recognize the expression included in the preview image according to a preset expression recognition model;

The photographing triggering module 203 is configured to trigger photographing to collect the user's image when it is detected that the user is at rest and the user's expression is the target expression.

A photographing device provided in an embodiment of the present application obtains a preview image of a user photographed in a shooting area by shooting in a preset photographing mode; and recognizes the preview image included in the preview image according to a preset expression recognition model Expression; when it is detected that the user is in a stationary state, and the user's expression is the target expression, a photo is triggered to collect an image of the user. By adopting the above technical solution, automatic photographing can be performed according to the expression of the user when the user is in a stationary state, and the automatic photographing can be achieved without the user manually triggering or relying on the photographing auxiliary tool, which can improve the convenience of photographing.

Optionally, the image acquisition module is specifically used for:

The expression recognition module is specifically used for:

Optionally, it also includes:

Optionally, the camera trigger module is specifically used to:

Trigger a photo to collect a second number of images of the user.

Optionally, it also includes:

A start prompting module for prompting shooting when the user is detected to be in a stationary state;

Also includes:

Optionally, it also includes:

The photo trigger module is specifically used for:

Optionally, it also includes:

A storage medium containing computer-executable instructions provided in an embodiment of the present application is not limited to the above-mentioned photographing operation, and can also perform related operations in the photographing method provided in any embodiment of the present application.

An embodiment of the present application further provides a storage medium containing computer-executable instructions, which when executed by a computer processor is used to execute a photographing method, the method includes:

Storage medium-any kind of memory device or storage device. The term "storage media" is intended to include: installation media such as CD-ROM, floppy disk or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc ; Non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements. The storage medium may also include other types of memory or a combination thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system through a network such as the Internet. The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (eg, in different computer systems connected through a network). The storage medium may store program instructions executable by one or more processors (eg, embodied as a computer program).

An embodiment of the present application provides a terminal device, and the terminal device may be integrated with the photographing device provided by the embodiment of the present application.

7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. An embodiment of the present application provides a terminal device 30, including a memory 31, a processor 32, and a computer stored on the memory 31 and capable of running on the processor. Program, when the processor executes the computer program, the photographing method described in the above embodiment is implemented. The terminal device provided by the embodiment of the present application can improve the convenience of taking pictures.

FIG. 8 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in FIG. 8, the terminal device may include: a casing (not shown in the figure), a touch screen (not shown in the figure), a touch button (not shown in the figure), a memory 301, and a central processing unit (Central Processing) Unit, CPU) 302 (also called processor, hereinafter referred to as CPU), circuit board (not shown in the figure) and power supply circuit (not shown in the figure). The circuit board is placed inside the space enclosed by the housing; the CPU 302 and the memory 301 are provided on the circuit board; the power circuit is used to supply power to each circuit or device of the terminal device The memory 301 is used to store executable program code; the CPU 302 runs the computer program corresponding to the executable program code by reading the executable program code stored in the memory 301 to achieve:

The terminal device further includes: a peripheral interface 303, an RF (Radio Frequency) circuit 305, an audio circuit 306, a speaker 311, a power management chip 308, an input / output (I / O) subsystem 309, a touch screen 312, other The input / control device 310 and the external port 304 communicate through one or more communication buses or signal lines 307.

It should be understood that the illustrated terminal device 300 is only an example of the terminal device, and the terminal device 300 may have more or fewer components than shown in the figure, and two or more components may be combined, Or it can have different component configurations. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

The following describes in detail the terminal device for photographing provided in this embodiment. The terminal device uses a mobile phone as an example.

Memory 301, which can be accessed by CPU 302, peripheral interface 303, etc. The memory 301 can include high-speed random access memory, and can also include non-volatile memory, such as one or more disk storage devices, flash memory devices , Or other volatile solid-state storage devices.

Peripheral interface 303, which can connect input and output peripherals of the device to CPU 302 and memory 301.

I / O subsystem 309, which can connect input and output peripherals on the device, such as touch screen 312 and other input / control devices 310, to peripheral interface 303. The I / O subsystem 309 may include a display controller 3091 and one or more input controllers 3092 for controlling other input / control devices 310. Among them, one or more input controllers 3092 receive electrical signals from other input / control devices 310 or send electrical signals to other input / control devices 310. Other input / control devices 310 may include physical buttons (press buttons, rocker buttons, etc. ), Dial pad, slide switch, joystick, click wheel. It is worth noting that the input controller 3092 can be connected to any of the following: a keyboard, an infrared port, a USB interface, and a pointing device such as a mouse.

The touch screen 312 is an input interface and an output interface between the user terminal device and the user, and displays the visual output to the user, and the visual output may include graphics, text, icons, video, and the like.

The display controller 3091 in the I / O subsystem 309 receives electrical signals from the touch screen 312 or sends electrical signals to the touch screen 312. The touch screen 312 detects the contact on the touch screen, and the display controller 3091 converts the detected contact into interaction with the user interface object displayed on the touch screen 312, that is, realizes human-computer interaction. Icons for games, icons connected to the corresponding network, etc. It is worth noting that the device may also include a light mouse, which is a touch-sensitive surface that does not display visual output or an extension of the touch-sensitive surface formed by a touch screen.

The RF circuit 305 is mainly used to establish communication between the mobile phone and the wireless network (that is, the network side), and realize data reception and transmission between the mobile phone and the wireless network. For example, sending and receiving short messages, e-mail, etc. Specifically, the RF circuit 305 receives and transmits RF signals, which are also called electromagnetic signals. The RF circuit 305 converts electrical signals into electromagnetic signals or converts electromagnetic signals into electrical signals, and communicates with the communication network and other devices through the electromagnetic signals Communicate. The RF circuit 305 may include known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, CODEC ( COder-DECoder (codec) chipset, subscriber identity module (Subscriber Identity Module, SIM), etc.

The audio circuit 306 is mainly used to receive audio data from the peripheral interface 303, convert the audio data into electrical signals, and send the electrical signals to the speaker 311.

The speaker 311 is used to restore the voice signal received by the mobile phone from the wireless network through the RF circuit 305 to a sound and play the sound to the user.

The power management chip 308 is used for power supply and power management for the hardware connected to the CPU 302, the I / O subsystem, and the peripheral interface.

The terminal device provided by the embodiment of the present application can improve the convenience of taking pictures.

The photographing apparatus, storage medium, and terminal device provided in the above embodiments can perform the photographing method provided in any embodiment of the present application, and have corresponding function modules and beneficial effects for performing the method. For technical details that are not described in detail in the above embodiments, please refer to the photographing method provided in any embodiment of the present application.

Note that the above are only the preferred embodiments of the present application and the technical principles applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application. The scope is determined by the scope of the appended claims.

Claims

A photographing method, including:

When shooting in the preset shooting mode, obtain a preview image of the user who is shot in the shooting area;

Identify the expression included in the preview image according to a preset expression recognition model;

When it is detected that the user is in a stationary state, and the user's expression is a target expression, a photograph is triggered to collect an image of the user.
The method according to claim 1, wherein the acquiring the preview image of the user captured in the shooting area includes:

Acquire the first number of preview images of the users photographed in the shooting area within a preset time;

The recognizing the expression included in the preview image according to the preset expression recognition model includes:

Identifying the expression included in the first number of preview images through a preset expression recognition model;

If the first number of preview images all include a target expression, it is determined that the user's expression is the target expression.
The method according to claim 2, wherein, within a preset time, after acquiring the first number of preview images of the users photographed in the shooting area, further comprising:

Detecting changes in pixels in the first number of preview images;

If the change of the pixel points meets the preset condition, it is determined that the user is in a stationary state.
The method of claim 1, wherein the triggering to take a picture to collect the image of the user comprises:

Trigger a photo to collect a second number of images of the user.
The method of claim 4, wherein, when it is detected that the user is in a stationary state, further comprising:

Prompt for shooting;

After triggering to take a picture to collect the image of the user, the method further includes:

Prompt for completion.
The method of claim 1, wherein before the triggering to take a picture to collect the image of the user, further comprising:

Determine the profile information and face angle of the user in the preview image;

Determine the user's photographing posture according to the profile information and the face angle;

When it is detected that the user is in a stationary state, and the user's expression is a target expression, triggering a photograph to collect an image of the user includes:

When it is detected that the user is in a stationary state, the expression of the user is a target expression, and the photographing posture is a preset posture, a photograph is triggered to collect images of the user.
The method of claim 1, wherein before identifying the expression included in the preview image according to a preset expression recognition model, further comprising:

Input the preset face data set into the preset model for training to obtain the preset expression recognition model;

Wherein, the preset face data set includes preset face images, face key point data and expressions, and the preset expression recognition model is used to identify face key point data in the input image, and according to the The face key point data determines the expression included in the input image.
A photographing device, including:

The image acquisition module is used to acquire the preview image of the user who is shot in the shooting area when shooting in the preset shooting mode;

An expression recognition module, configured to recognize the expression included in the preview image according to a preset expression recognition model;

The photographing triggering module is used to trigger the photographing to collect the user's image when it is detected that the user is at rest and the user's expression is the target expression.
The photographing device according to claim 8, wherein the image acquisition module is further used to:

Acquire the first number of preview images of the users photographed in the shooting area within a preset time;

The expression recognition module is specifically used for:

Identifying the expression included in the first number of preview images through a preset expression recognition model;

If the first number of preview images all include a target expression, it is determined that the user's expression is the target expression.
The photographing device according to claim 9, wherein the image acquisition module is further used to:

Within a preset time, after acquiring a first number of preview images of the users photographed in the shooting area, detecting a change in pixels in the first number of preview images;

If the change of the pixel points meets the preset condition, it is determined that the user is in a stationary state.
The photographing device according to claim 8, wherein the image acquisition module is further used to:

Trigger a photo to collect a second number of images of the user.
The photographing device according to claim 8, wherein the device further comprises:

A start prompting module for prompting shooting when the user is detected to be in a stationary state;

The completion prompting module is used for prompting completion after triggering the photographing to collect the image of the user.
The photographing device according to claim 8, further comprising:

The information determining module is used to determine the profile information and face angle of the user in the preview image before the triggering of the photograph to collect the image of the user;

A posture determination module, configured to determine the user's photo posture based on the profile information and the face angle;

The photo trigger module is specifically used for:

When it is detected that the user is in a stationary state, the expression of the user is a target expression, and the photographing posture is a preset posture, a photograph is triggered to collect images of the user.
The photographing device according to claim 8, further comprising:

The model training module is used to input the preset face data set into the preset model for training before identifying the expression included in the preview image according to the preset expression recognition model to obtain the preset expression recognition model ;

Wherein, the preset face data set includes preset face images, face key point data and expressions, and the preset expression recognition model is used to identify face key point data in the input image, and according to the The face key point data determines the expression included in the input image.
A computer-readable storage medium on which a computer program is stored, wherein when the program is executed by a processor, the photographing method according to any one of claims 1-7 is implemented.
A terminal device, including a memory, a processor, and a computer program stored on the memory and executable by the processor, the processor implementing the computer program realizes:

When shooting in the preset shooting mode, obtain a preview image of the user who is shot in the shooting area;

Identify the expression included in the preview image according to a preset expression recognition model;

When it is detected that the user is in a stationary state, and the user's expression is a target expression, a photograph is triggered to collect an image of the user.
The terminal device of claim 16, wherein the processor is further configured to execute:

Acquire the first number of preview images of the users photographed in the shooting area within a preset time;

The recognizing the expression included in the preview image according to the preset expression recognition model includes:

Identifying the expression included in the first number of preview images through a preset expression recognition model;

If the first number of preview images all include a target expression, it is determined that the user's expression is the target expression.
The terminal device according to claim 17, wherein the processor is further configured to execute:

Detecting changes in pixels in the first number of preview images;

If the change of the pixel points meets the preset condition, it is determined that the user is in a stationary state.
The terminal device of claim 16, wherein the processor is further configured to execute:

Determine the profile information and face angle of the user in the preview image;

Determine the user's photographing posture according to the profile information and the face angle;

When it is detected that the user is in a stationary state, and the user's expression is a target expression, triggering a photograph to collect an image of the user includes:

When it is detected that the user is in a stationary state, the expression of the user is a target expression, and the photographing posture is a preset posture, a photograph is triggered to collect images of the user.
The terminal device of claim 16, wherein the processor is further configured to execute:

Input the preset face data set into the preset model for training to obtain the preset expression recognition model;

Wherein, the preset face data set includes preset face images, face key point data and expressions, and the preset expression recognition model is used to identify face key point data in the input image, and according to the The face key point data determines the expression included in the input image.