WO2020134891A1 - Photo previewing method for electronic device, graphical user interface and electronic device - Google Patents

Abstract

A photo previewing method for an electronic device, a graphical user interface and an electronic device, the method comprising: adjusting a body image of a photographed individual during photo previewing or video previewing so that the body represented by the adjusted body image is beautified compared to the actual body of the photographed individual. Body beautification may comprise: beautifying body proportions (for example, lengthening the legs, and widening the shoulders), and adjusting the size of the body (for example, thinning the waist, thinning the legs, thinning the stomach and thinning the hips or enlarging parts of the body). The described solution enables a user to beautify the body of a photographed individual during image capture previewing, and operation is intuitive, simple and effective.

Description

Photo preview method of electronic device, graphical user interface and electronic device

This application requires the priority of the Chinese patent application submitted to the State Intellectual Property Office of China on December 26, 2018, with the application number 201811608420.2 and the invention titled "Photo Preview Method for Electronic Devices, Graphic User Interface, and Electronic Devices", all of which are The content is incorporated into this application by reference.

Technical field

The invention relates to the field of electronic technology, in particular to a photo preview method applied to electronic devices, a graphical user interface, and electronic devices.

Background technique

With the popularity of social networks, there is an increasing demand for smart portrait beautification in the market, and a variety of beauty cameras are constantly emerging. However, portable electronic devices (such as mobile phones and tablet computers) equipped with beautification functions are still not widely used. Existing beautification applications such as Photoshop have high requirements on computing resources and are not suitable for deployment on portable electronic devices. Moreover, the user interaction design of existing beautification applications such as Photoshop is not intuitive and requires professionals to use it more skillfully, and the operation is more complicated and cumbersome.

Summary of the invention

The object of the present invention is to provide a photo preview method, graphic user interface (GUI) and electronic device of an electronic device, which can enable the user to beautify the person being photographed during the photo preview process, and the operation is intuitive and simple Effective, can improve the utilization rate of electronic equipment.

The above and other objectives will be achieved by the features in the independent claims. Further implementations are reflected in the dependent claims, the description and the drawings.

In a first aspect, a photo preview method for an electronic device is provided. The electronic device may have a 3D camera module. The method may include: the electronic device may turn on the 3D camera module, and collect the first image of the photographed person through the 3D camera module. The electronic device may display a first graphical user interface, and the first graphical user interface may include a preview frame, and the preview frame displays the first image of the photographed person. The electronic device may detect the first user operation, and in response to the first user operation, may display the second image of the photographed person in the preview box of the first graphical user interface.

Among them, the contour of the first body part of the photographed person in the second image is adjusted according to the first body parameters, and the contour of the second body part of the photographed person in the second image is adjusted according to the second body parameters, The first part is different from the second part, and the first body parameters are different from the second body parameters. That is to say, the electronic device can adjust the contours of different body parts in the first image by using different body parameters.

With reference to the first aspect, in a possible implementation manner, the first graphical user interface may further include a first control, and the first user operation is a user operation for selecting a beauty level that acts on the first control; the selected The higher the body level, the greater the gap between the contour of the first part in the second image and the contour of the first part in the first image, and the greater the gap between the contour of the second part in the second image and the contour of the second part in the first image .

Specifically, the first user operation may specifically include a second user operation acting on the first control (such as a rightward sliding operation on the first control), and a third user operation acting on the first control (such as the first control Swipe left). specific:

The electronic device can detect a second user operation acting on the first control (such as a rightward sliding operation on the first control). The second user operation can be used to increase the body level corresponding to the person being photographed (such as increasing from body level 7 To body level 10). In response to the second user operation, the electronic device may refresh the display content in the preview box. The body shape of the photographed person's body image in the preview frame after refreshing has a beautification compared with the body shape of the photographed person's body image in the preview frame before refreshing. That is to say, in response to the second user operation, the outline of the first part in the second image in the preview frame after refreshing is compared with the outline of the first part in the second image in the preview frame before refreshing, and The gap between the contours of the first part is larger. Compared with the contour of the second part in the second image in the preview frame before refreshing, the contour of the second part in the second image in the preview frame after refreshing is larger than the contour of the second part in the first image.

The electronic device can detect a third user operation acting on the first control (such as a leftward sliding operation on the first control). The third user operation can be used to reduce the body level corresponding to the person being photographed (such as from the body level 10 Up to body level 7), in response to the operation of the third user, the display content in the preview box can be refreshed. The body shape of the photographed person's body image in the preview frame after refreshing is closer to the actual body shape of the photographed person than that in the preview frame before refreshing.

That is to say, in response to the third user operation, the outline of the first part in the second image in the preview frame after refreshing is compared with the outline of the first part in the second image in the preview frame before refreshing, and The gap between the contours of the first part is smaller. The contour of the second part in the second image in the preview frame after refreshing is smaller than the contour of the second part in the second image in the preview frame before refreshing, and the contour of the second part in the first image is smaller.

With reference to the first aspect, in a possible implementation manner, when it is detected that the person being photographed is a pregnant woman, the electronic device may display a prompt message (such as the text "pregnant woman detected") in the preview box. The photographed person is a pregnant woman. At this time, the electronic device can use the beauty parameters corresponding to the selected beauty level to perform beauty treatment on the color images of other partial bodies of the photographed person (such as arms, legs, and shoulders), but not the stomach of the photographed person. The image is processed with a body to preserve the characteristics of the pregnant woman. That is to say, the outline of the belly of the photographed person in the second image displayed in the preview frame matches the outline of the belly of the photographed person in the first image.

With reference to the first aspect, in a possible implementation manner, the electronic device may adjust the color value of the face skin of the person photographed in the second image in response to the detected fourth user operation. The fourth user operation may be a user operation that turns on the “beauty skin” function and selects a specific skin beauty level (such as skin beauty level 7).

Specifically, when the body image of the photographed person displayed in the preview box has been subjected to body treatment using the body parameters corresponding to the selected body level, and the fourth user operation is detected, the electronic device may refresh the display in the preview box Content, the image of the photographed person displayed in the refreshed preview box includes a face image and a body image, where the face image is a face image after skin treatment, and the body image is corresponding to the selected beauty level Body image The body image that has undergone body treatment. That is to say, the user can continue to beautify the skin of the photographed person on the basis of the beautification of the photographed person to achieve a more comprehensive beautification effect. Conversely, the user can continue to beautify the photographed person on the basis of the skin beautification of the photographed person.

With reference to the first aspect, in a possible implementation, the electronic device may also display the second control in the preview box. In response to the detected user operation of pressing and holding on the second control without releasing, the electronic device may display the first image in the preview box. In response to the detected user operation to release the second control, the electronic device may display the second image in the preview box. In this way, the user can compare the body shape represented by the first image with the body shape represented by the second image.

With reference to the first aspect, in a possible implementation manner, the electronic device may display the first prompt information in the preview box for prompting the contour of the first part in the second image and the contour of the first part in the first image , The difference between the outline of the second part in the second image and the outline of the second part in the first image. The second image is the image of the photographed person after the body treatment displayed in the preview frame, and the first image is the image of the photographed person collected by the 3D camera module. The contours of the body parts of the captured person in the first image are consistent with the actual contours. These differences can be expressed as the difference in waist circumference, the difference in leg circumference, the difference in leg length, the difference in shoulder width and so on.

With reference to the first aspect, in a possible implementation manner, the first prompt information may also be used to prompt one or more of the following manifestations: weight to be lost, required amount of exercise, and calories to be consumed. The weight that needs to be lost is used to indicate how much weight the person to be photographed needs to lose to approach or reach the body shape represented by the body image displayed in the preview box. The required amount of exercise is used to indicate how much exercise the person to be photographed needs to approach or reach the figure represented by the body image displayed in the preview box. The calories that need to be consumed are used to remind how much calories the person to be photographed needs to approach or reach the body shape shown in the preview frame.

With reference to the first aspect, in a possible implementation, if the 3D camera module is a rear-mounted 3D camera module, the electronic device displays the third control and the second prompt message in the preview box. The third control can be used To monitor user operations for sharing fitness information, the second prompt information can be used to prompt the user to click the third control to share fitness information with the person being photographed. In response to the detected user operation acting on the third control, the electronic device may display a user interface for sharing fitness information. In response to the detected user operation of selecting a contact in the user interface for sharing fitness information to share fitness information, the electronic device may share fitness information with the selected contact.

With reference to the first aspect, in a possible implementation, if the 3D camera module is a front-mounted 3D camera module, the electronic device may display the fourth control and the third prompt information in the preview box. The third prompt information It can be used to prompt the user to click the fourth control to view fitness information. In response to the detected user operation acting on the fourth control, the electronic device may display fitness information.

With reference to the first aspect, in a possible implementation manner, the first graphical user interface may further include: a shooting control. The electronic device may save the second image displayed in the preview frame in response to the detected user operation acting on the shooting control.

With reference to the first aspect, in a possible implementation, the first graphical user interface may further include: a shooting mode list, the shooting mode list includes multiple shooting mode options; the multiple shooting modes include: the first shooting mode option and the first Two shooting mode options. among them:

When the first shooting mode option is in the selected state, the shooting control is specifically used to monitor the user operation that triggers the photographing, and in response to the detected user operation acting on the shooting control, saving the second image displayed in the preview box, specifically including : In response to the detected user operation acting on the shooting control, save the second image displayed in the preview box as a photo.

When the second shooting mode option is selected, the shooting control is specifically used to monitor the user operation that triggers the recording. In response to the detected user operation acting on the shooting control, the second image displayed in the preview box is saved, specifically including : In response to the detected user operation acting on the shooting control, save the second image displayed in the preview box as a video.

With reference to the first aspect, in a possible implementation manner, before displaying the second image of the photographed person in the preview box of the first graphical user interface in response to the first user operation, the electronic device may also be based on the photographed person Of the images identify human bone points.

With reference to the first aspect, in a possible implementation manner, before displaying the second image of the photographed person in the preview box of the first graphical user interface in response to the first user operation, the electronic device may further determine the photographed person The angle with the plane where the electronic device is located, and the angle does not exceed the preset angle threshold.

With reference to the first aspect, in a possible implementation manner, before the second image of the photographed person is displayed in the preview box of the first graphical user interface in response to the first user operation, the electronic device may also according to the photographed person The angle with the plane where the electronic device is located determines the perspective transformation matrix, and performs perspective transformation on the first image of the person to be photographed according to the perspective transformation matrix.

In a second aspect, a photo preview method for an electronic device is provided. The electronic device may have a 3D camera module. The method may include: the electronic device turns on the 3D camera module, and may collect the image of the person being photographed through the 3D camera module. The electronic device can display a second graphical user interface. The second graphical user interface includes: a preview box and a body template column; wherein, the preview box displays a color image collected by the 3D camera module, and the body template column displays one or more bodies Template options.

In response to the detected user operation acting on the figure template option, the electronic device may display the fifth control.

In response to the detected first user operation acting on the fifth control, the electronic device may refresh the preview box. The body image of the photographed person's body image in the preview box after refreshing is closer to the body type represented by the selected body template than that of the body image of the photographed person in the preview box before refreshing.

In response to the detected second user operation acting on the fifth control, the electronic device can refresh the preview frame; the body image of the photographed person in the preview frame after refreshing is compared with the body shape of the preview frame before refreshing The body shape of the photographed person's body image is closer to the actual shape of the person being photographed.

In a third aspect, an electronic device is also provided. The electronic device may include a 3D camera module, a display screen, a touch sensor, a wireless communication module, a memory, and one or more processors. The one or more processors are used to execute One or more computer programs stored in the aforementioned memory, wherein:

The 3D camera module can be used to collect the first image of the person being photographed;

The display screen may be used to display a first graphical user interface, and the first user interface may include a preview frame, and the preview frame displays a first image of the photographed person collected by the 3D camera module.

The touch sensor may be used to detect the first user operation.

The display screen may be used to display the second image of the person being photographed in the preview box in response to the first operation.

Among them, the contour of the first body part of the photographed person in the second image is adjusted according to the first body parameters, and the contour of the second body part of the photographed person in the second image is adjusted according to the second body parameters, The first part is different from the second part, and the first body parameters are different from the second body parameters. That is to say, the electronic device can adjust the contours of different body parts in the first image by using different body parameters.

For a specific implementation manner of each component included in the electronic device in the third aspect, reference may be made to the photo preview method described in the first aspect, and details are not described herein again.

According to a fourth aspect, an electronic device is also provided. The electronic device may include an apparatus, which may implement any possible implementation manner as in the first aspect, or as any possible implementation manner in the second aspect.

In a fifth aspect, there is also provided a photo preview device, which has a function to implement the behavior of an electronic device in the above method. The above-mentioned functions can be realized by hardware, and can also be realized by hardware executing corresponding software. The above hardware or software includes one or more modules corresponding to the above functions.

According to a sixth aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the computer program, the computer device is realized As in any possible implementation manner in the first aspect, or as in any possible implementation manner in the second aspect.

According to a seventh aspect, a computer program product containing instructions is characterized in that, when the computer program product runs on an electronic device, the electronic device is caused to execute any possible implementation manner as in the first aspect, or as in the second Any possible implementation in any aspect.

In an eighth aspect, a computer-readable storage medium is provided, which includes instructions, characterized in that, when the above instructions run on an electronic device, the above electronic device executes any possible implementation manner as in the first aspect, or as in Any possible implementation in the two aspects.

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions in the embodiments of the present application, the drawings required in the embodiments of the present application will be described below.

FIG. 1A is a schematic diagram of a structure of an electronic device provided by an embodiment;

1B is a schematic diagram of a software structure of an electronic device provided by an embodiment;

2A is a schematic diagram of a user interface for an application menu on an electronic device provided by an embodiment;

2B is a schematic diagram of a rear 3D camera module on an electronic device provided by an embodiment;

3A-3C are schematic diagrams of shooting scenes involved in this application;

3D-3E are schematic diagrams of a user interface of a character beautification function provided by an embodiment;

4A-4H are schematic diagrams of a UI for beautifying a person being photographed during a photo preview provided by an embodiment;

5A-5D are schematic diagrams of a UI provided for revoking a beautified body during a photo preview provided by an embodiment;

6A-6G are schematic diagrams of a UI for pushing fitness information during a photo preview provided by an embodiment;

7A-7H are schematic diagrams of UIs for pushing fitness information during a photo preview provided by another embodiment;

8A-8B are schematic diagrams of UI provided before and after beautification of a comparison body during a photo preview provided by an embodiment;

9A-9C are schematic diagrams of UIs for users to take photos provided by an embodiment;

10A-10G are schematic diagrams of a UI for beautifying a photographed person during video preview provided by an embodiment;

11A-11E are schematic diagrams of a UI for beautifying a photographed person using a body shape template provided by an embodiment;

12A-12C are schematic diagrams of a one-button UI for full-body beautification provided by an embodiment;

13A-13D are schematic diagrams of UI for beautifying multiple people provided by an embodiment;

14A-14B are schematic diagrams of a UI for manual beautification provided by an embodiment;

15A-15D are schematic diagrams of UI for beautifying the figures in the captured pictures provided by an embodiment;

FIG. 16 is a schematic flowchart of a photo preview method provided by an embodiment;

17A-17C show pixel points in a color image, a depth image, and a 3D coordinate space, respectively;

Figure 18 shows basic human bone points;

FIG. 19 is a schematic diagram of calculating the angle α between the photographed person and the electronic device according to the depth value of the bone point and the 2D coordinates;

20 is a schematic diagram of calculating the length of the bone between the bone points according to the depth value of the bone points and the 2D coordinates;

21 is a schematic diagram of dividing a color image collected by a 3D camera module into a color image (that is, a foreground image) and a background image of a person to be photographed;

Fig. 22 is a comparison diagram before and after image restoration;

FIG. 23 is an architectural diagram of functional modules included in an electronic device according to an embodiment.

detailed description

The terminology used in the following embodiments of the present application is only for the purpose of describing specific embodiments, and is not intended to be a limitation of the present application. As used in the specification and appended claims of this application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also Includes plural expressions unless the context clearly indicates otherwise. It should also be understood that the term "and/or" used in this application refers to and includes any or all possible combinations of one or more of the listed items.

The following describes an electronic device, a user interface for such an electronic device, and an embodiment for using such an electronic device. In some embodiments, the electronic device may be a portable electronic device that also contains other functions such as personal digital assistant and/or music player functions, such as a mobile phone, a tablet computer, and a wearable electronic device with wireless communication function (such as a smart watch) Wait. Exemplary embodiments of portable electronic devices include, but are not limited to

Or portable electronic devices of other operating systems. The above portable electronic device may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface or a touch panel. It should also be understood that in some other embodiments, the above electronic device may not be a portable electronic device, but a desktop computer with a touch-sensitive surface or a touch panel.

The term “user interface (UI)” in the specification, claims, and drawings of this application is a media interface for interaction and information exchange between applications or operating systems and users, which implements the internal form of information And users can accept the conversion between forms. The user interface of the application is source code written in a specific computer language such as java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that can be recognized by the user. Such as pictures, text, buttons and other controls. Controls (also called widgets) are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, and scroll bars. (scrollbar), pictures and text. The attributes and contents of the controls in the interface are defined by tags or nodes. For example, XML uses <Textview>, <ImgView>, <VideoView> and other nodes to specify the controls contained in the interface. A node corresponds to a control or attribute in the interface, and the node is rendered as user-visible content after being parsed and rendered. In addition, many applications, such as hybrid applications (hybrid application) interface usually also contains web pages. A web page, also known as a page, can be understood as a special control embedded in the application interface. The web page is source code written in a specific computer language, such as hypertext markup language (hypertext markuplanguage, GTML), cascading styles Tables (cascading styles, CSS), java scripts (JavaScript, JS), etc., web page source code can be loaded and displayed as user-recognizable content by the browser or a web page display component similar to the browser function. The specific content contained in a web page is also defined by tags or nodes in the source code of the web page. For example, GTML uses <p>, <img>, <video>, and <canvas> to define the elements and attributes of the web page.

The common form of user interface is the graphical user interface (graphic user interface, GUI), which refers to the user interface related to computer operation displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of the electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. Visual interface elements.

The following embodiments of the present application provide a photo preview method for an electronic device, a graphical user interface, and an electronic device, which can enable the user to beautify the person being photographed during the camera preview process, and the operation is intuitive and simple and effective, which can improve the electronic device Of usage.

In the following embodiments of the present application, the application "camera" of an electronic device such as a smartphone can provide two kinds of beautification functions: a "beauty" function and a "beauty" function. among them,

The "beauty" function can be used to adjust the face image of the person being photographed during the photo preview or video preview process, so that the face represented by the adjusted face image occurs compared to the actual face of the person being photographed Skin beautification, such as skin whitening, skin dermabrasion (such as removing acne, freckles, wrinkles, etc.), and so on. The adjustment of the face image involved in the "beauty" function may refer to the smoothing of the face image using algorithms such as surface blur, mean filtering, and bilateral filtering. In the following embodiments of the present application, such processing performed on the face image may be referred to as skin beautification processing.

The "Beauty" function can be used to adjust the body image of the person being photographed during the photo preview or video preview process, so that the body shape of the adjusted body image is beautified compared to the actual body shape of the person being photographed. Body beautification can include: beautifying the body proportions (such as lengthening the legs, widening the shoulders, etc.), adjusting the body's fat and thin (such as thin waist, thin legs, thin belly, thin buttocks, or full body treatment, etc.) ). The adjustment of the body image involved in the "beauty" function may include: determining the target position to which the bone point needs to be adjusted according to the beautification parameters of the beautification ratio of the body (such as the body parameters corresponding to the elongated legs, which will be introduced in the following content) Common image scaling algorithms such as bicubic, bilinear, and nearest neighbor can be used to scale the local body image between bone points, so that the bone point can be located at its corresponding target position after the local body image is scaled. In order to achieve the purpose of beautifying the proportion of the body. The adjustment of the body image involved in the "beauty" function may also include: using common image scaling algorithms such as bicubic, bilinear, and nearest neighbor to scale the overall body image or partial body image of the person being photographed (scale) Treatment to achieve the purpose of adjusting body fat or thin. For example, the image processing related to stovepipe may include performing compression processing on the leg image using an image scaling algorithm, and the leg image represented by the compressed leg image is slimmer than the actual leg of the person being photographed. For another example, the image processing of the waist shaping design may include: using an image scaling algorithm to compress the middle portion of the waist image, and stretching the upper and lower ends of the waist image. After the image processing, the waist image expresses The waist is more curved than the actual waist of the person being photographed, and the waist represented by this image-processed waist image may be an S-shaped waist (the waist is thinner in the middle). In the following embodiments of the present application, such processing performed on the body image may be referred to as body treatment.

In the following embodiments of the present application, body images (including images of various partial bodies, such as leg images) targeted for body treatment refer to color images of the body collected by the color camera module. Similarly, the face image targeted by the beauty treatment also refers to the color image of the face collected by the color camera module.

In the following embodiments of the present application, the "Body" function analyzes the color images collected by the color camera module of the electronic device based on the bone point recognition technology to identify the bone points of the person being photographed, such as the head point, neck point, left shoulder point, and right Shoulder point. The following content will introduce the skeleton point recognition technology and the basic skeleton points of the human body, which will not be repeated here. Further combined with the depth data (including bone point depth data) collected by the depth camera module of the electronic device for the person being photographed, the electronic device can analyze the body proportion, fat and thin body shape of the person being photographed. Then, the electronic device can adjust the body image between the bone points of the photographed person during the camera preview, so that the figure represented by the adjusted body image is more beautified than the actual figure of the figure. In this way, the photographer can beautify the photographed person during the camera preview (photo preview and video preview), the operation is intuitive, simple and effective, and the utilization rate of the electronic device can be improved.

In the following embodiments of the present application, the "beauty" function and/or the "beauty" function may be integrated into the "portrait" camera function and video recording function included in the "camera" application. The "beauty" function and/or "beauty" function can also be used as a separate camera function in the "camera" application. "Portrait" camera function is a camera function that is set when the subject is a person to highlight the person and enhance the beauty of the person in the captured picture. When the electronic device turns on the "portrait" camera function, the electronic device can use a larger aperture to keep the depth of field shallower to highlight the person, and can improve the color effect to optimize the person's skin color. When it is detected that the ambient light intensity is below a certain threshold, the electronic device can also turn on the flash to perform illumination compensation.

"Camera" is an application for capturing images on smart phones, tablets, and other electronic devices. The application does not limit the name of the application. The "portrait" camera function and video recording function may be the camera function included in the "camera" application. In addition, the "camera" application can also include a variety of other camera functions. The camera parameters such as aperture size, shutter speed, and sensitivity corresponding to different camera functions can be different, and different camera effects can be presented. The camera function can also be called a camera mode, for example, the "portrait" camera function can also be called a "portrait" camera mode.

It can be understood that “beauty skin”, “beauty body”, and “portrait” are just some of the words used in this embodiment, and the meanings they represent have been recorded in this embodiment, and their names do not constitute any limitation to this embodiment. . In addition, in some other embodiments of the present application, “beauty skin” may also be referred to as other terms such as “face beauty”. Similarly, the "beauty body" mentioned in the embodiments of the present application may also be called other names such as "slimming shape" in other embodiments.

First, an exemplary electronic device 100 provided in the following embodiments of the present application is introduced.

FIG. 1A shows a schematic structural diagram of the electronic device 100.

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, key 190, motor 191, indicator 192, 3D camera module 193, display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180G, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a central processing unit (CPU), and a graphics processing unit (GPU). , Neural network processor (neural-network processing unit, NPU), modem processor, image signal processor (ISP), controller, memory, video codec, digital signal processor (digital signal processor) processor, DSP), baseband processor, etc. Among them, the different processing units may be independent devices or may be integrated in one or more processors. In some embodiments, the electronic device 100 may also include one or more processors 110.

The controller may be the nerve center and command center of the electronic device 100. The controller can generate the operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetch and execution.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. The repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the electronic device 100.

In some embodiments, the processor 110 may include one or more interfaces. Interfaces can include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit, sound, I2S) interface, pulse code modulation (pulse code modulation (PCM) interface, universal asynchronous transceiver (universal) asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and /Or universal serial bus (USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, the processor 110 may include multiple sets of I2C buses. The processor 110 may respectively couple the touch sensor 180K, charger, flash, 3D camera module 193, etc. through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface, and realize the touch function of the electronic device 100.

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, to realize the function of answering the phone call through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface to implement the function of answering the phone call through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the 3D camera module 193. MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI) and so on. In some embodiments, the processor 110 and the 3D camera module 193 communicate through a CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to realize the display function of the electronic device 100.

The GPIO interface can be configured via software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 to the 3D camera module 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface that conforms to the USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present invention is only a schematic description, and does not constitute a limitation on the structure of the electronic device 100. In other embodiments, the electronic device 100 may also use different interface connection methods in the foregoing embodiments, or a combination of multiple interface connection methods.

The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive the charging input of the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through the wireless charging coil of the electronic device 100. While the charging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140, and supplies power to the processor 110, internal memory 121, external memory, display screen 194, 3D camera module 193, and wireless communication module 160. The power management module 141 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters. In some other embodiments, the power management module 141 may also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the electronic device 100 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), and the like. The mobile communication module 150 can receive the electromagnetic wave from the antenna 1 and filter, amplify, etc. the received electromagnetic wave, and transmit it to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it to electromagnetic wave radiation through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a high-frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110, and may be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (bluetooth, BT), and global navigation satellites that are applied to the electronic device 100 Wireless communication solutions such as global navigation (satellite system, GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (infrared, IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives the electromagnetic wave via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and convert it to electromagnetic waves through the antenna 2 to radiate it out. Exemplarily, the wireless communication module 160 may include a Bluetooth module, a Wi-Fi module, and the like.

In some embodiments, the antenna 1 of the electronic device 100 and the mobile communication module 150 are coupled, and the antenna 2 and the wireless communication module 160 are coupled so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global mobile communication system (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long-term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidou navigation system (BDS), and a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite-based augmentation system (SBAS).

The electronic device 100 can realize a display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations, and is used for graphics rendering. The processor 110 may include one or more GPUs that execute instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel can use a liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active matrix organic light-emitting diode (active-matrix organic light-emitting diode) emitting diode, AMOLED, flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The electronic device 100 can realize a camera function through a 3D camera module 193, an ISP, a video codec, a GPU, a display screen 194, an application processor AP, and a neural network processor NPU.

The 3D camera module 193 can be used to collect color image data and depth data of the subject. The ISP can be used to process the color image data collected by the 3D camera module 193. For example, when taking a picture, the shutter is opened, and light is transmitted to the photosensitive element of the camera through the lens, and the optical signal is converted into an electrical signal. The photosensitive element of the camera transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. ISP can also optimize the algorithm of image noise, brightness and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be set in the 3D camera module 193.

In some embodiments, the 3D camera module 193 may be composed of a color camera module and a 3D sensing module.

In some embodiments, the photosensitive element of the camera of the color camera module may be a charge coupled device (charge), a coupled device (CCD), or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals.

In some embodiments, the 3D sensing module may be a (time of flight, TOF) 3D sensing module or a structured light (structured light) 3D sensing module. Among them, the structured light 3D sensing is an active depth sensing technology. The basic components of the structured light 3D sensing module may include an infrared (Infrared) emitter, an IR camera module, and the like. The working principle of the structured light 3D sensing module is to first emit a specific pattern of spots on the object to be photographed, and then receive the light spot pattern encoding on the surface of the object, and then compare the similarities and differences with the original projected spot. And use the principle of triangle to calculate the three-dimensional coordinates of the object. The three-dimensional coordinates include the distance between the electronic device 100 and the object to be photographed. Among them, TOF 3D sensing is also an active depth sensing technology. The basic components of TOF 3D sensing modules may include infrared (Infrared) transmitters, IR camera modules, and so on. The working principle of the TOF 3D sensing module is to calculate the distance (that is, the depth) between the TOF 3D sensing module and the object to be captured by the time of infrared foldback to obtain a 3D depth map.

The structured light 3D sensing module can also be used in the fields of face recognition, somatosensory game consoles, industrial machine vision inspection, etc. The TOF 3D sensing module can also be used in game consoles, augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) and other fields.

In other embodiments, the 3D camera module 193 may also be composed of two or more cameras. The two or more cameras may include a color camera, and the color camera may be used to collect color image data of the object to be photographed. The two or more cameras may use stereo vision technology to collect depth data of the object being photographed. Stereo vision technology is based on the principle of parallax of human eyes. Under natural light, two or more cameras are used to capture images of the same object from different angles, and then triangulation and other operations are performed to obtain the electronic device 100 and the The distance information between the objects is the depth information.

In some embodiments, the electronic device 100 may include 1 or N 3D camera modules 193, where N is a positive integer greater than 1. Specifically, the electronic device 100 may include a front 3D camera module 193 and a rear 3D camera module 193. Among them, the front 3D camera module 193 can usually be used to collect the color image data and depth data of the photographer facing the display screen 194, and the rear 3D camera module can be used to collect the subject (such as people) faced by the photographer , Landscape, etc.) color image data and depth data.

In some embodiments, the CPU or GPU or NPU in the processor 110 can process the color image data and depth data collected by the 3D camera module 193. In some embodiments, the NPU can recognize the color image collected by the 3D camera module 193 (specifically, the color camera module) through a neural network algorithm based on the bone point recognition technology, such as a convolutional neural network algorithm (CNN) Data to determine the bone points of the person being photographed. The CPU or GPU can also run a neural network algorithm to determine the bone point of the person to be photographed based on the color image data. In some embodiments, the CPU, GPU, or NPU can also be used to confirm the figure of the person being photographed based on the depth data collected by the 3D camera module 193 (specifically, the 3D sensing module) and the identified bone points (such as The proportion of the body, the fatness and thinness of the body parts between the bone points), and can further determine the body beautification parameters for the photographed person, and finally process the photographed image of the photographed person according to the body beautification parameters to make the shooting The figure in the image is beautified. Subsequent embodiments will introduce in detail how to perform body treatment on the image of the captured person based on the color image data and depth data collected by the 3D camera module 193, which will not be repeated here.

The digital signal processor is used to process digital signals. In addition to digital image signals, it can also process other digital signals. For example, when the electronic device 100 is selected at a frequency point, the digital signal processor is used to perform Fourier transform on the energy at the frequency point.

The video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (MPEG)-1, MPEG-2, MPEG-3, MPEG-4, etc.

NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information and can continue to self-learn. The NPU can realize applications such as intelligent recognition of the electronic device 100, such as image recognition, face recognition, voice recognition, and text understanding.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, photos, videos and other data in an external memory card.

The internal memory 121 may be used to store one or more computer programs including instructions. The processor 110 may run the above instructions stored in the internal memory 121, so that the electronic device 100 executes the photo preview method of the electronic device provided in some embodiments of the present application, as well as various functional applications and data processing. The internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area can store the operating system; the storage program area can also store one or more application programs (such as gallery, contacts, etc.) and so on. The storage data area may store data (such as photos, contacts, etc.) created during use of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and so on.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, and an application processor. For example, music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also known as "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or voice message, the voice can be received by bringing the receiver 170B close to the ear.

The microphone 170C, also known as "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through the human mouth, and input the sound signal to the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C. In addition to collecting sound signals, it may also implement a noise reduction function. In other embodiments, the electronic device 100 may also be provided with three, four, or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The headset interface 170D is used to connect wired headsets. The earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile electronic device (open mobile terminal) platform (OMTP) standard interface, and the American Telecommunications Industry Association (cellular telecommunications industry association of the United States, CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. The capacitive pressure sensor may be a parallel plate including at least two conductive materials. When force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the strength of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch position but have different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the movement posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the jitter angle of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the jitter of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a clamshell machine, the electronic device 100 may detect the opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the holster or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to recognize the posture of electronic devices, and can be used in horizontal and vertical screen switching, pedometer and other applications.

The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting scenes, the electronic device 100 may use the distance sensor 180F to measure distance to achieve fast focusing.

The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touch.

The fingerprint sensor 180G is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access to application lock, fingerprint photo taking, fingerprint answering call, and the like.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 performs to reduce the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In some other embodiments, when the temperature is below another threshold, the electronic device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

The touch sensor 180K can also be called a touch panel or a touch-sensitive surface. The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 constitute a touch screen, also called a "touch screen". The touch sensor 180K is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. The visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100, which is different from the location where the display screen 194 is located.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive a blood pressure beating signal. In some embodiments, the bone conduction sensor 180M may also be provided in the earphone and combined into a bone conduction earphone. The audio module 170 may parse out the voice signal based on the vibration signal of the vibrating bone block of the voice part acquired by the bone conduction sensor 180M to realize the voice function. The application processor may analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M to implement the heart rate detection function.

The key 190 includes a power-on key, a volume key, and the like. The key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100.

The motor 191 may generate a vibration prompt. The motor 191 can be used for vibration notification of incoming calls and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, audio playback, etc.) may correspond to different vibration feedback effects. For the touch operation of different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminder, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects. Touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the amount of power change, and can also be used to indicate messages, missed calls, notifications, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. The same SIM card interface 195 can insert multiple cards at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through a SIM card to realize functions such as call and data communication. In some embodiments, the electronic device 100 uses eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

The electronic device 100 exemplarily shown in FIG. 1A may display various user interfaces described in the following embodiments through the display screen 194. The electronic device 100 can detect a touch operation in each user interface through the touch sensor 180K, for example, a click operation in each user interface (such as a touch operation on an icon, a double-click operation), and, for example, an upward or in each user interface. Swipe down, or perform a gesture of drawing a circle, and so on. In some embodiments, the electronic device 100 may detect a motion gesture performed by the user holding the electronic device 100, such as shaking the electronic device, through the gyro sensor 180B, the acceleration sensor 180E, or the like. In some embodiments, the electronic device 100 can detect non-touch gesture operations through the 3D camera module 193 (eg, 3D camera, depth camera).

The software system of the electronic device 100 may adopt a layered architecture, event-driven architecture, micro-core architecture, micro-service architecture, or cloud architecture. The embodiment of the present invention takes the Android system with a layered architecture as an example to exemplarily explain the software structure of the electronic device 100.

FIG. 1B is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present invention.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, from top to bottom are the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer.

The application layer may include a series of application packages.

As shown in FIG. 1B, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

The application framework layer provides an application programming interface (application programming interface) and programming framework for applications at the application layer. The application framework layer includes some predefined functions.

As shown in FIG. 1B, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and so on.

The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, intercept the screen, etc.

Content providers are used to store and retrieve data, and make these data accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc.

The view system includes visual controls, such as controls for displaying text and controls for displaying pictures. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including an SMS notification icon may include a view to display text and a view to display pictures.

The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call status (including connection, hang up, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloading, message reminders, etc. The notification manager can also be a notification that appears in the status bar at the top of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, the text message is displayed in the status bar, a prompt sound is emitted, the electronic device vibrates, and the indicator light flashes.

Android Runtime includes core library and virtual machine. Android runtime is responsible for the scheduling and management of the Android system.

The core library contains two parts: one part is the function function that Java language needs to call, and the other part is the core library of Android.

The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

The system library may include multiple functional modules. For example: surface manager (surface manager), media library (Media library), 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.

The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, G.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to realize 3D graphics drawing, image rendering, synthesis, and layer processing.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

The software system shown in Figure 1B involves the presentation of applications that use sharing capabilities (such as a gallery, file manager), an instant sharing module that provides sharing capabilities, a print service (print service) and a print spooler (print background service) that provide print capabilities , And the application framework layer provide printing framework, WLAN services, Bluetooth services, and the core and the bottom layer provide WLAN Bluetooth capabilities and basic communication protocols.

The following describes the workflow of the software and hardware of the electronic device 100 in combination with capturing a photographing scene.

When the touch sensor 180K receives the touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes touch operations into original input events (including touch coordinates, time stamps and other information of touch operations). The original input event is stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer, and identifies the control corresponding to the input event. Taking the touch operation as a touch operation, the control corresponding to the touch operation is a camera application icon as an example. The camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer, and the 3D camera Module 193 captures still images or video.

The following describes an exemplary user interface for the application menu on the electronic device 100.

FIG. 2A exemplarily shows an exemplary user interface 21 on the electronic device 100 for an application menu. As shown in FIG. 2A, the electronic device 100 may be configured with a 3D camera module 193. In some embodiments, 193-1 may be a color camera, and 193-2 may be a structured light 3D camera module. In other embodiments, 193-1 may be a color camera, and 193-2 may be a TOF 3D camera module. In still other embodiments, 193-1 and 193-2 may be two color cameras. As shown in FIG. 2A, the 3D camera module 193 may be disposed on the top of the electronic device 100, such as the "bangs" position of the electronic device 100 (ie, the area AA shown in FIG. 2A). It can be known that, in addition to the 3D camera module 193, the area AA may also include an illuminator 197 (not shown in FIG. 1A), a speaker 170A, a proximity light sensor 180G, an ambient light sensor 180L, and the like. In some embodiments, as shown in FIG. 2B, the back of the electronic device 100 may also be equipped with a 3D camera module 193 and an illuminator 197.

As shown in FIG. 2A, the user interface 21 may include a status bar 201, a tray 223 with frequently used application icons, a calendar indicator 203, a weather indicator 205, a navigation bar 225, and other application icons. among them:

The status bar 201 may include: one or more signal strength indicators 201-1 of the mobile communication signal (also called a cellular signal), an indicator 201-2 of the operator of the mobile communication signal, a time indicator 201-3, Battery status indicator 201-4, etc.

The calendar indicator 203 can be used to indicate the current time, such as date, day of the week, hour and minute information, and so on.

The weather indicator 205 can be used to indicate the weather type, such as cloudy to sunny, light rain, etc., and can also be used to indicate temperature and other information.

The tray 223 with frequently used application icons may display: a phone icon 223-1, a short message icon 223-2, a contact icon 221-4, and the like.

The navigation bar 225 may include: a return button 225-1, a main screen (Gome) button 225-3, an outgoing task history button 225-5, and other system navigation keys. When it is detected that the user clicks the return button 225-1, the electronic device 100 may display the previous page of the current page. When it is detected that the user clicks the main interface button 225-3, the electronic device 100 may display the main interface. When it is detected that the user clicks the outgoing task history button 225-5, the electronic device 100 may display the task recently opened by the user. The naming of each navigation key can be other, and this application does not limit it. Not limited to virtual keys, each navigation key in the navigation bar 225 may also be implemented as a physical key.

Other application icons may be, for example: Wechat icon 211, QQ icon 212, Twitter icon 213, Facebook icon 214, mailbox icon 215, cloud sharing icon 216, memo Icon 217, set icon 218, gallery icon 219, camera icon 220. The user interface 21 may also include a page indicator 221. Other application icons may be distributed on multiple pages, and the page indicator 221 may be used to indicate which page of the application the user is currently browsing. The user can slide the area of other application icons left and right to browse the application icons in other pages.

In some embodiments, the user interface 21 exemplarily shown in FIG. 2A may be a main interface (Gome screen).

In some other embodiments, the electronic device 100 may further include a home screen key. The home screen key may be a physical key or a virtual key (such as key 225-3). The home screen key can be used to receive user instructions and return the currently displayed UI to the home interface, which can facilitate the user to view the home screen at any time. The above instruction may specifically be an operation instruction in which the user presses the home screen key once, or an operation instruction in which the user presses the home screen key twice in a short period of time, or a long press of the home screen key in a predetermined time Operating instructions. In some other embodiments of the present application, the home screen key may also be integrated with a fingerprint reader, so that when the home screen key is pressed, fingerprint collection and identification are performed accordingly.

It can be understood that FIG. 2A only exemplarily shows the user interface on the electronic device 100, and should not constitute a limitation on the embodiments of the present application.

The following describes an application scenario involved in this application: an image shooting scenario.

As shown in FIG. 3A, the electronic device may detect a touch operation (such as a click operation on the icon 220) acting on the icon 220 of the camera, and in response to the operation, the user interface 31 exemplarily shown in FIG. 3B may be displayed. The user interface 31 may be a user interface of a "camera" application, and may be used for a user to take a picture, for example, take a picture or record a video. "Camera" is an application for capturing images on smart phones, tablets, and other electronic devices. The application does not limit the name of the application. That is to say, the user can click the icon 220 to open the user interface 31 of the "camera". Not limited to this, the user may also open the user interface 31 in other applications, for example, the user clicks the shooting control in “WeChat” to open the user interface 31. "WeChat" is a social application that allows users to share photos taken with others.

FIG. 3B exemplarily shows a user interface 31 of a “camera” application on an electronic device such as a smartphone. As shown in FIG. 3B, the user interface 31 may include an area 301, a shooting mode list 302, a control 303, a control 304, and a control 305. among them:

The area 301 may be referred to as a preview frame 301. The preview frame 301 can be used to display the color image collected by the 3D camera module 193 in real time. The electronic device can refresh the display content in real time, so that the user can preview the color image currently collected by the 3D camera module 193. Here, the 3D camera module 193 may be a rear camera or a front camera.

One or more shooting mode options may be displayed in the shooting mode list 302. The one or more camera options may include: night view mode option 302A, portrait mode option 302B, camera mode option 302C, video mode option 302D, and more shooting mode options 302E. The one or more camera options can be represented as text information on the interface, for example, night view mode option 302A, portrait mode option 302B, camera mode option 302C, video mode option 302D, and more shooting mode options 302E can correspond to the text "night view" , "Portrait", "Take Photo", "Video", "More". Not limited to this, the one or more camera options can also be represented as icons or other forms of interactive elements (IE) on the interface. In some embodiments, the electronic device 100 may select the photographing mode option 302C by default, and the display state of the photographing mode option 302C (eg, the photographing mode option 302C is highlighted) may indicate that the photographing mode option 302C has been selected.

The electronic device 100 may detect a user operation acting on a shooting mode option, and the user operation may be used to select a shooting mode, and in response to the operation, the electronic device 100 may turn on the shooting mode selected by the user. In particular, when the user operation acts on more shooting mode options 302E, the electronic device 100 can further display more other shooting mode options, such as large aperture shooting mode options, slow motion shooting mode options, etc., which can be shown to the user More abundant camera functions. Not limited to that shown in FIG. 3B, more shooting mode options 302E may not be displayed in the shooting mode list 302, and the user may browse other shooting mode options by swiping left/right in the shooting mode list 302.

The control 303 can be used to monitor user operations that trigger shooting (photographing or video recording). The electronic device can detect a user operation on the control 303 (such as a click operation on the control 303), and in response to the operation, the electronic device 100 can save the image in the preview box 301. The saved image can be a picture or a video. In addition, the electronic device 100 can also display the thumbnail of the saved image in the control 304. That is to say, the user can click the control 303 to trigger shooting. The control 303 may be a button or other forms of controls. In this application, the control 303 may be referred to as a shooting control.

The control 304 can be used to monitor user operations that trigger camera switching. The electronic device 100 can detect a user operation on the control 304 (such as a click operation on the control 304), and in response to the operation, the electronic device 100 can switch the camera (such as switching the rear camera to the front camera, or the front Set the camera to switch to the rear camera).

The control 305 can be used to monitor user operations that trigger the opening of the "Gallery". The electronic device 100 can detect a user operation (such as a click operation on the control 305) acting on the control 305, and in response to the operation, the electronic device 100 can display a user interface of "Gallery", in which the electronic device can be displayed 100 saved pictures. Here, "Gallery" is an application for managing pictures on smart phones, tablet computers, and other electronic devices, and may also be called an "album." In this embodiment, the name of the application is not limited. "Gallery" can support users to perform various operations on the pictures stored on the electronic device, such as browsing, editing, deleting, selecting and other operations.

It can be seen that the user interface 31 can show the user various camera functions (modes) provided by the “camera”, and the user can choose to turn on the corresponding shooting mode by clicking the shooting mode option.

Based on the above image shooting scene, some embodiments of a user interface (UI) implemented on the electronic device 100 are described below.

FIG. 3C exemplarily shows the user interface 32 provided by the “portrait” photographing function of the “camera” application.

In the shooting mode list 302, the electronic device 100 can detect a user operation acting on the portrait mode option 302B (such as a click operation on the portrait mode option 302B), and in response to the user operation, the electronic device 100 can turn on the "portrait" photo Function and display the user interface shown as an example in FIG. 3C. The definition of the electronic device 100 enabling the "portrait" camera function has been explained in the foregoing, and will not be repeated here. In this application, the portrait mode option may be referred to as the first shooting mode option.

As shown in FIG. 3C, the user interface 32 includes: a preview box 301, a shooting mode list 302, a control 303, a control 304, a control 305, and 306, a control 207. Among them: for the preview box 301, the shooting mode list 302, the control 303, the control 304, and the control 305, reference may be made to the relevant description in the user interface 31, which will not be repeated here. The control 306 can be used to monitor the user operation of opening the light effect template option, and the control 307 can be used to monitor the user operation of opening the character beautification option.

When a user operation acting on the control 306 (such as a click operation on the control 306) is detected, the electronic device 100 may display various light effect template options in the user interface 31. Different light effect templates can represent (or correspond to) different light effect parameters, such as light source position, layer fusion parameter, texture pattern projection position, projection direction, etc. The user can select different light effect templates to make the photos taken show different effects. This application does not limit the interface representation forms of various light effect template options in the user interface 31.

When a user operation (such as a click operation on the control 307) acting on the control 307 is detected, the electronic device 100 may display the user interface 33 exemplarily shown in FIG. 3D. FIG. 3D exemplarily shows the user interface provided by the character beautification function. The user interface shown by way of example in FIG. 3D will be described in detail in the following content, which will not be repeated here.

In some embodiments, in response to a user operation acting on the portrait mode option 302B, the electronic device 100 may also update the display state of the portrait shooting mode option, and the updated display state may indicate that the portrait shooting mode has been selected. For example, the updated display state may be the text information "Portrait" corresponding to the highlight shooting mode option 303B. Not limited to this, the updated display state can also present other interface expressions, such as the font of the text information "Portrait" becomes larger, the text information "Portrait" is framed, the text information "Portrait" is underlined, options 303B color deepening, etc.

In some embodiments, after the electronic device 100 turns on the "portrait" camera function, if the electronic device 100 does not detect a person in the color image collected from the 3D camera module 193, a prompt message 308 may be output in the preview box 301 In this case, the prompt information 308 may be the text “No person detected”, which may be used to alert the electronic device 100 that no person has been detected.

As can be seen from FIG. 3C, the character beautification function can be integrated into the "portrait" camera function. Not limited to this, the person beautification function may also be a camera function in the “camera” application. At this time, the person beautification mode option may be displayed in the shooting mode list 302 in the user interface 31. In response to the user operation acting on the character beautification mode option, the electronic device 100 may display the user interface provided by the character beautification function exemplarily shown in FIGS. 3D-3E.

3D-3E exemplarily show the user interface 33 provided by the character beautification function of the "camera" application.

As shown in FIGS. 3D-3E, the user interface 33 includes a preview frame 301, a shooting mode list 302, a control 303, a control 304, a control 305, and a skin beauty option 309 and a body beauty option 310. Among them: for the preview box 301, the shooting mode list 302, the control 303, the control 304, and the control 305, reference may be made to the relevant description in the user interface 31, which will not be repeated here.

The skin beauty option 309 and the body beauty option 310 can be represented as icons on the interface, as shown in FIG. 3D. Not limited to icons, the skin beauty option 309 and the body beauty option 310 can also be expressed as text (such as text “beauty”, “beauty”) or other forms of interactive elements (IE) on the interface.

When the electronic device 100 detects a user operation acting on the skin beautification option 309 (such as a click operation on the skin beautification option 309), as shown in FIG. 3D, the electronic device 100 may display the control 311 in the user interface 33, and may Turn on the "beauty" function. The "beauty" function can be used to perform skin beautification on the face image of the person being photographed in the photo preview or video preview process, so that the face represented by the face image after skin treatment is compared with the actual face of the person being photographed Skin beautification has occurred on the face, such as skin whitening, skin dermabrasion (such as removing acne, freckles, wrinkles, etc.), and so on. The control 311 can be used by the user to adjust the degree of skin beauty of the person being photographed. Here, the degree of beautification may refer to the degree of beautification of the skin of the face of the photographed person. The degree of skin beautification can also be called the skin beautification level. For example, there can be a total of 11 skin-beauty levels from skin-beauty level 0 to skin-beauty level 10. The higher the skin-beauty level, the higher the beautification of the skin; the skin-beauty level 0 can mean that the face of the person being photographed is not skin-beautified. The face image of the photographed person displayed in the preview frame 301 does not show skin beautification compared to the actual face of the photographed person; the beautification level 10 can represent a greater degree of the face of the photographed person Skin beautification, at this time the face represented by the face image of the photographed person displayed in the preview frame 301 has a greater degree of skin beautification than the actual face of the photographed person.

When the electronic device 100 detects a user operation (such as a click operation on the body option 310) acting on the body option 310, the user operation is used to select the body option 310. As shown in FIG. 3E, the electronic device 100 may The control 312 is displayed in the interface 33, and the "Body" function can be turned on. The "Beauty" function can be used to perform body treatment on the body image of the person being photographed during the photo preview or video preview process, so that the body image after the body treatment performs a body beautification compared to the actual body shape of the person being photographed. The control 312 may be a slider. The control 312 can be used by the user to adjust the beauty of the person being photographed. Here, the degree of beautification may refer to the degree of beautification of the body of the person being photographed. The body level can also be called the body level. For example, there can be a total of 11 beauty levels from body level 0 to body level 10. The higher the body level, the higher the degree of beautification of the body; the body level 0 can indicate that the body image of the person being photographed is not subject to body treatment. The body image of the photographed person displayed in the preview box 301 is consistent with the actual body shape of the photographed person, that is, no beautification occurs; the body level 10 can represent a greater degree of body beautification of the body of the photographed person. The body image of the photographed person displayed in the preview frame 301 has a greater degree of body beautification than the actual shape of the photographed person.

In some embodiments, the skin beauty option 309 may be the option selected by default. That is, when a user operation for turning on the character beautification function is detected, the electronic device 100 may display the user interface shown in FIG. 3D by default. In other embodiments, the body style option 310 may be the option selected by default. That is, when a user operation for turning on the character beautification function is detected, the electronic device 100 may display the user interface shown in FIG. 3E by default. Here, the user operation for turning on the character beautification function may be the aforementioned user operation acting on the portrait mode option 302B, and the aforementioned user operation acting on the character beautification mode option.

In some embodiments, when the beauty option 309 is selected (selected by default or selected by the user), the display state of the beauty option 309 may indicate that the beauty option 309 has been selected; when the beauty option 310 is selected When selected by default or selected by the user, the display state of the body style option 310 may indicate that the body style option 310 has been selected. For example, the display state in which the skin beauty option 309 (or body beauty option 310) has been selected may be the highlight skin beauty option 309 (or body beauty option 310). Not limited to this, the display state may also present other interface expressions.

In some embodiments, after the electronic device 100 enables the “beauty skin” function, if the electronic device 100 does not detect a person from the color image collected by the 3D camera module 193, a prompt message 308 may be output in the preview box 301, The prompt information 308 may be the text “No person detected”, which may be used to prompt the electronic device 100 that no person is detected. Specifically, the electronic device 100 can detect whether the color image collected by the 3D camera module 193 includes a face image. If the face image is included, it is determined that a person is detected, otherwise it is determined that no person is detected.

In some embodiments, after the electronic device 100 turns on the “Body” function, if the electronic device 100 does not detect a person from the color image collected by the 3D camera module 193, a prompt message 308 may be output in the preview box 301, prompting The information 308 may be the text "No person detected", which may be used to prompt the electronic device 100 that no person has been detected. Specifically, the electronic device 100 may analyze whether the color image collected by the 3D camera module 193 contains human bone points based on the bone point recognition technology. If the human bone points are included, it is determined that a person is detected, otherwise it is determined that no person is detected. The specific implementation of determining human bone points based on bone point recognition technology will be described in detail in the subsequent content, and will not be expanded here first.

FIGS. 4A-4G exemplarily show a UI embodiment for beautifying a photographed person during a photo preview. In the embodiment of FIGS. 4A-4E, the 3D camera module 193 can collect the image of the person 411. The image of the person 411 may be an image of the front body of the person 411 or an image of the back body of the person 411. The image of the person 411 may include a body image and a face image. The body image may include images of multiple partial bodies (eg, legs, waist, shoulders, stomach, etc.).

In the embodiment of FIGS. 4A-4C, if the human bone point of the person 411 is recognized from the collected color image based on the bone point recognition technology, the electronic device 100 may determine that the person has been detected and may be used to detect A user operation of performing a body treatment on the image of the person photographed in the preview frame 301. This user operation may be referred to as a first user operation. In response to the first user operation, the electronic device 100 may update the display content in the preview frame 301, and display the image of the photographed person after the body treatment in the preview frame 301. The image of the photographed person after the body treatment displayed in the preview frame 301 may be referred to as a second image. The image of the captured person (that is, the foreground image) collected by the 3D camera module displayed in the preview frame 301 may be referred to as a first image.

Among them, the contour of the first body part of the photographed person in the second image can be adjusted according to the first body parameters, and the contour of the second body part of the photographed person in the second image can be adjusted according to the second body parameters For adjustment, the first part and the second part may be different, and the first body parameter and the second body parameter may be different. That is to say, the electronic device 100 can use different body parameters to adjust the contours of different body parts in the first image to obtain the contours of different body parts in the second image.

Specifically, the first user operation may specifically include a second user operation acting on the control 312 (such as a rightward sliding operation on the control 312), and a third user operation acting on the control 312 (such as a leftward operation on the control 312) Slide operation). specific:

As shown in FIG. 4A, when the beauty level selected for the person 411 is the body level 0, the electronic device 100 does not perform the body treatment on the body image of the person 411. At this time, the figure represented by the body image of the person 411 displayed in the preview frame 301 matches the actual figure of the person 411. At this time, the body image of the person 411 displayed in the preview frame 301 is the first image of the captured person.

As shown in FIG. 4A, the electronic device 100 can detect a second user operation (such as a rightward sliding operation on the control 312) acting on the control 312, and the second user operation can be used to increase the body level corresponding to the character 411 (such as from Body level 7 is increased to body level 10). In response to the second user operation, the electronic device 100 may refresh the display content in the preview box 301. The body image of the person 411 in the preview frame 301 after refreshing is more beautified than the body image of the person 411 in the preview frame 301 before refreshing.

Among them, body beautification can include one or more of the following: beautification of body proportions (such as elongated legs, widened shoulders, etc.), and the whole body or partial body (such as waist, legs, stomach, shoulders, etc.) Fat and thin to adjust. Beautifying the body ratio can refer to adjusting the length and width of each part of the body according to the human body aesthetics, so that the adjusted body ratio is closer to the standard body ratio defined by the human body aesthetics, such as the golden ratio of the upper and lower body (with the navel as the boundary, upper The lower body ratio is about 5 to 8). The beautification of body proportions can also be not limited to human body aesthetics. For example, the person to be photographed that satisfies the golden ratio of the upper and lower body can still stretch his legs to show the advantage of stature with protruding legs.

That is, in response to the detected second user operation, the outline of the first part in the second image in the preview frame after refreshing is compared with the outline of the first part in the second image in the preview frame before refreshing, and The difference in the outline of the first part in the first image is greater. Compared with the contour of the second part in the second image in the preview frame before refreshing, the contour of the second part in the second image in the preview frame after refreshing is larger than the contour of the second part in the first image.

As shown in FIG. 4A, the electronic device 100 can detect a third user operation acting on the control 312 (such as a leftward sliding operation on the control 312), and the third user operation can be used to lower the body level corresponding to the character 411 (such as from The body level 10 is reduced to the body level 7), in response to the third user operation, the display content in the preview box 301 can be refreshed. The body image of the person 411 in the preview frame 301 after refreshing is closer to the actual body shape of the person 411 than the body image of the person 411 in the preview frame 301 before refreshing.

That is, in response to the detected third user operation, the outline of the first part in the second image in the preview frame after refreshing is compared with the outline of the first part in the second image in the preview frame before refreshing, and The difference in the outline of the first part in the first image is smaller. The contour of the second part in the second image in the preview frame after refreshing is smaller than the contour of the second part in the second image in the preview frame before refreshing, and the contour of the second part in the first image is smaller.

In some embodiments, in addition to body level 0, body level 5 or other body levels may be the default body level of the “Body” function, so that the user can click the “Body option 310” to open the “Body” function and see in the preview box 301 To the image of the person who has been processed by the body, the user can directly click on the control 303 to save the image of the person 411 that has been processed by the body. The operation is simple and the use efficiency is high. In particular, if the default body level is body level 5, which is in the middle of the control 312, then the sliding distance required by the user to manually adjust the body level to the left or right can be greatly reduced, which can save user operations and improve the use efficiency.

As can be seen from FIG. 4A, during the process of the user sliding from the left to the right on the control 312, the electronic device 100 can successively detect multiple user operations to improve the body level, and can refresh the display content in the preview box 301 multiple times, so that Compared with the body image before the refresh, the body image after each refresh has a body beautification, which can provide a visual effect of gradually enhancing the body beautification in the preview box 301. In addition, when the user slides from right to left on the control 312, the electronic device 100 can successively detect multiple user operations that lower the body level, and can refresh the display content in the preview box 301 multiple times. Provides a visual effect that gradually diminishes beautification.

In some embodiments, in response to the second user operation or the third user operation, both the third user operation or the third user operation are used to select a body shape level for the photographed person 411, and the electronic device 100 may utilize the selected body level ( For example, the beauty parameters corresponding to the beauty level 5) perform a body treatment on the body image of the person 411 collected by the 3D camera module 193, so as to obtain the body image of the person 411 displayed in the refreshed preview frame 301. Regarding how the electronic device uses the beauty parameters corresponding to the selected beauty level to perform the body treatment on the body image of the person 411, details will be described in the subsequent method embodiments, and details are not described here.

In some embodiments, as shown in FIGS. 4D-4E, when it is detected that the person 411 is a pregnant woman, the electronic device 100 may display a prompt message (such as the text "pregnant woman detected") in the preview box 301, the prompt message may be Prompt character 411 is a pregnant woman. At this time, as shown in FIGS. 4D-4E, the electronic device 100 can use the beauty parameters corresponding to the selected beauty level to perform beautification on the color images of other partial bodies of the person 411 (such as arms, legs, and shoulders) Processing without performing body treatment on the belly image of the person 411 to retain the characteristics of the pregnant woman. That is to say, the outline of the belly of the photographed person in the second image displayed in the preview frame 301 matches the outline of the belly of the photographed person in the first image. Optionally, when detecting that the person 411 is a pregnant woman, the electronic device 100 may also display a switch control in the preview box 301. When the switch control is in the "on" state, it means that the characteristics of pregnant women are retained, that is, the electronic device 100 does not perform body treatment on the belly image; when the switch control is in the "off" state, it means that the characteristics of the pregnant woman are not retained, that is, the electronic device 100 can be Fig. 4A shows body contouring on the belly image to show the effect of thin belly.

In some embodiments, when it is detected that the person 411 changes the posture or the person being photographed is changed, the electronic device 100 may still use the body parameters corresponding to the selected body level to perform body treatment on the current person being photographed.

In some embodiments, as shown in FIGS. 4D-4E, when the body image of the person 411 displayed in the preview box 301 has been subjected to body treatment using the body parameters corresponding to the selected body level, and it is detected Skin” function and select a specific skin beauty level (such as skin beauty level 7) user operation, the electronic device 100 can refresh the display content in the preview box 301, the image of the person 411 displayed in the refreshed preview box 301 includes the face The image and the body image, wherein the face image is a face image after skin treatment, and the body image is a body image that has undergone body treatment using the body parameters corresponding to the selected body level. That is to say, the user can continue to beautify the skin of the photographed person on the basis of the beautification of the photographed person to achieve a more comprehensive beautification effect. Conversely, the user can continue to beautify the photographed person on the basis of the skin beautification of the photographed person.

It is not limited to scenes where the person being photographed is shot in front or back, as shown in FIGS. 4F-4H, when the posture of the person being captured by the 3D camera module 193 is other postures (such as standing sideways or sitting sideways, etc.), The electronic device 100 may also perform a beautification process on the body image of the photographed person in response to the user operation for selecting the body-beauty level, so as to realize the beautification of the photographed person in other poses.

FIGS. 5A-5D exemplarily show a UI embodiment in which the beautification is cancelled during the photo preview. Among them, FIGS. 5A-5B exemplarily show a UI embodiment for revoking a partial body; FIG. 5C-5D exemplarily show a UI embodiment for revoking a whole-body body with one click.

As shown in FIG. 5A, in the human body image displayed in the preview frame 301, the partial body where the figure 411 has been beautified includes: left arm, right arm, stomach, left leg, and right leg. These partial bodies were respectively beautified as follows: thin left arm, thin right arm, thin belly, elongated left leg, elongated right leg. In the preview box 301, the electronic device 100 may display the undo icon 501 on the images of these partial beautified bodies. In this way, the user can click the undo icon 501 on the partial body image to undo the beautification of the partial body.

As shown in FIG. 5B, the electronic device 100 can detect a user operation of the undo icon 501 (such as a click operation on the undo icon 501) acting on the belly image of the person 411. In response to this operation, the electronic device 100 can refresh the display content in the preview box 301, the shape of the belly represented by the belly image of the person 411 in the preview box 301 after refreshing matches the actual shape of the belly of the person 411, and the preview before refreshing The belly image of the character 411 in the frame 301 is thinner than the actual belly of the character 411.

That is, the electronic device 100 can detect a user operation (such as a click operation on the undo icon 501) of the undo icon 501 acting on the partial body image of the person being photographed. The partial body image is the selected partial body image. In response to this operation, the electronic device 100 may refresh the preview box 301. The shape of the partial body represented by the selected partial body image in the preview box 301 after refreshing is consistent with the actual shape of the partial body of the person being photographed, and the representation of the selected partial body image in the preview box 301 before refreshing Compared to the actual part of the person being photographed, the part of the body has been beautified.

The embodiments of FIGS. 5A-5B can support the user to cancel the body treatment of the partial body of the person being photographed in the preview box 301, so that the user can retain the original features of certain body parts (such as a sturdy and thick arm), which can be more Flexible to meet the user's body needs.

As shown in FIG. 5C, in the human body image displayed in the preview frame 301, the figure 411 has been beautified. In the preview box 301, the electronic device 100 may display the one-key undo icon 503 in the preview box 301. In this way, the user can click the one-key undo icon 503 to undo all body treatments performed on the entire body.

As shown in FIG. 5D, the electronic device 100 can detect a user operation (such as a click operation on the icon 503) acting on the one-key undo icon 503. In response to this operation, the electronic device 100 can refresh the display content in the preview box 301. The body image of the person 411 in the preview box 301 after refreshing shows the same shape as the actual figure of the person 411, while the preview box 301 before refreshing The figure represented by the body image of the person 411 is beautified compared to the actual figure of the person 411.

The embodiments of FIGS. 5C-5D can support the user to cancel the whole body body performed on the photographed person in the preview box 301 with one key, the operation is simple and effective, and the use efficiency can be improved.

6A-6G exemplarily show some UI embodiments for pushing fitness information during a photo preview. Fitness information can include fitness classes, fitness methods, fitness diet, etc. In this application, fitness information may be referred to as the first content. Among them, FIGS. 6A-6D exemplarily show UI embodiments for pushing fitness information when the rear camera module shoots; FIGS. 6E-6G exemplarily show UI implementations for pushing fitness information when the front camera module shoots example.

In the embodiments of FIGS. 6A-6D, the image of the person 411 displayed in the preview frame 301 is collected by the rear 3D camera module 193 of the electronic device 100. At this time, the person 411 is usually not a photographer holding the electronic device 100, but a friend, relative, etc. of the photographer. The photographer can share the fitness information with the photographed person 411 to help or encourage the photographed person 411 to exercise.

As shown in FIG. 6A, the electronic device 100 can display the control 601 and the prompt information 603 in the preview box 301. The control 601 can be used to monitor user operations that share fitness information. The prompt information 603 can be used to prompt the user to click the control 601 to share fitness information such as fitness classes, fitness methods, fitness diets, and the like with the characters 411. In some embodiments, the prompt information 603 may gradually disappear in the preview box 301 after being displayed for a period of time (eg, 1 second). The user can also cancel the display of the prompt information 603 by swiping up or left on the prompt information 603.

As shown in FIGS. 6B-6D, the electronic device 100 can detect a user operation (such as a click operation on the control 601) acting on the control 601. In response to this operation, the electronic device 100 may display a user interface 605 for sharing fitness information. The user can select a contact in the user interface 605 to share fitness information with the contact. The electronic device 100 may detect a user operation of selecting a contact to share fitness information in the user interface 605. In response to this operation, the electronic device 100 can share fitness information with the selected contact.

For example, the user interface 605 may be exemplarily shown in FIG. 6C. The user interface 605 may be a recent chat interface of the application "WeChat", and the conversation list 605-1 in the interface may be used to display a conversation in which one or more conversations have recently occurred. entry. These multiple conversations are associated with different contacts, such as "MAC", "Kate", "William", etc. The electronic device 100 can detect a user operation of selecting a contact, such as a click operation on the conversation item of the user and the contact "Kate", and in response to the operation, the electronic device 100 can share fitness information with the selected contact "Kate" , And can display the user and contact "Kate" dialogue interface 607. The dialogue interface 607 may be exemplarily shown in FIG. 6D, in which a message 607-1 sent by the user to the contact "Kate" may be displayed. Message 607-1 may include a brief description of fitness information. The message 607-1 may also be associated with a link to fitness information, which may be provided by a third-party fitness application. In this way, the contact "Kate" can click the message 607-1 to open the link after receiving the message 607-1, so that he can jump to the user interface of the third-party fitness application to view fitness information.

Not limited to the application "WeChat", the user interface 605 for sharing fitness information can also be the user interface of other social applications or other data sharing functions (such as Apple's AirDrop, Huawei's Huawei Share (Guawei, Share, etc.) provides a user interface for sharing data. In this way, it is easy to promote the fitness information, and it is convenient for the photographed person to obtain fitness information, which can encourage or help the photographed person to exercise.

In the embodiments of FIGS. 6E-6G, the image of the person 411 displayed in the preview frame 301 is collected by the front-mounted 3D camera module 193 of the electronic device 100. At this time, the person 411 may generally be (or include) a photographer holding the electronic device 100.

As shown in FIG. 6E, the electronic device 100 can display the control 608 and the prompt information 606 in the preview box 301. The prompt information 606 can be used to prompt the user to click the control 608 to view fitness information. Different from the embodiments of FIGS. 6A-6D, the control 608 in the preview box 301 can be used to monitor user operations that open fitness information instead of sharing fitness information. In this way, it is convenient for the photographer to obtain fitness information such as fitness classes, fitness guidance, and fitness diet plans when taking a self-portrait, thereby improving the user experience.

As shown in FIGS. 6F-6G, the electronic device 100 can detect a user operation (such as a click operation on the control 608) acting on the control 608. In response to this operation, the electronic device 100 can display fitness information. The fitness information may be specifically displayed in the user interface 609 exemplarily shown in FIG. 6G. The user interface 609 may be a user interface of a third-party fitness application program (such as "ABC Fitness"), and the specific implementation of the user interface is not limited in this application.

In the embodiments of FIGS. 6A-6D or FIGS. 6E-6G, the fitness information may be related to the body-level selected for the person being photographed. For example, the higher the body-level, the higher the difficulty of the fitness class. The fitness information can also be related to the gender, age and other characteristics of the person being photographed. For example, if the person being photographed is a female, the fitness information can be a fitness class suitable for women. Fitness information can also be related to the current shape of the person being photographed. For example, the fatter the person being photographed, the more difficult the fitness of the fitness class.

In the embodiments of FIGS. 6A-6D or 6E-6G, the electronic device 100 may specifically display the control 601 or the control 608 in the preview box 301 in the following situations:

Case 1: After detecting that the user selects the beauty level, the time for keeping the selected beauty level unchanged does not exceed a specific duration (such as 1 second).

Case 2: It is detected that the user clicks the control 303 under the selected beauty level to trigger shooting, and the selected beauty level may be a beauty level other than the beauty level 0.

Case 3: It is detected that the figure of the person being photographed is not a standard figure.

Among them, both case 1 and case 2 can indicate that the user is satisfied with the body shape of the photographed person presented in the preview box 301 at the selected beauty level. Not limited to the above cases, the electronic device 100 may also display the control 601 or the control 608 in the preview box 301 in other cases, so that the photographer can share fitness information with the person being photographed or the photographer can open the fitness information by himself.

7A-7G exemplarily show other UI embodiments for pushing fitness information during a photo preview. Among them, FIGS. 7A-7D exemplarily show UI embodiments for pushing fitness information when shooting with the rear camera module; FIGS. 7E-7G exemplarily show UI implementations for pushing fitness information when shooting with the front camera module example.

7A-7G embodiments can refer to FIG. 6A-6G embodiments. Different from the embodiment of FIGS. 6A-6G, the electronic device 100 may display prompt information 703 in the preview box 301.

The prompt information 703 can be used to prompt the difference between the contour of the first part in the second image and the contour of the first part in the first image, the second part in the second image The difference between the outline of and the outline of the second part in the first image. Wherein, the second image is the image of the photographed person after the body treatment displayed in the preview frame 301, and the first image is the image of the photographed person collected by the 3D camera module. The contours of the body parts of the photographed person in the first image are consistent with the actual contours. These differences can be expressed as: differences in waist circumference, differences in leg circumference, differences in leg length, differences in shoulder width and so on.

The prompt information 703 can also be used to prompt one or more of the following manifestations: weight to be lost, amount of exercise required, calories to be consumed, and so on. The weight that needs to be lost is used to prompt how much weight the person to be photographed needs to lose to approach or reach the body shape represented by the body image displayed in the preview box 301. The required amount of exercise is used to indicate how much exercise the person to be photographed needs to approach or reach the figure represented by the body image displayed in the preview box 301. The calories that need to be consumed are used to indicate how much calories the person to be shot needs to approach or reach the body shape represented by the body image displayed in the preview box 301. For example, the gap can be the text "You still need to be away from your ideal size: weight loss 5KG aerobic run 100KM consumes 7000 calories". This makes it easier for users to understand fitness goals and encourages users to exercise to achieve their ideal body shape. Here, the ideal body shape may be the body shape displayed in the preview box 301 at the currently selected beauty level.

Among them, in the embodiment of FIGS. 7A-7D, the prompt information 703-1 can also be displayed in the prompt information 703. The prompt information 703-1 can be used to prompt the user to click the prompt information 703-1 to share fitness information with the photographed person 411. The electronic device can detect a user operation acting on the prompt information 703-1 (such as a click operation on the prompt information 703-1). In response to this operation, the electronic device 100 may display a user interface 705 for sharing fitness information. The user can select a contact in the user interface 705 to share fitness information to the contact. The electronic device 100 may detect a user operation of selecting a contact to share fitness information in the user interface 705. In response to this operation, the electronic device 100 can share fitness information with the selected contact. For details, reference may be made to the embodiments in FIGS. 6A to 6D, and details are not described herein again.

Among them, in the embodiment of FIGS. 7E-7G, the prompt information 706 may also be displayed in the prompt information 706. The prompt information 706-1 can be used to prompt the user to click the prompt information 706-1 to share fitness information with the person 411 to be photographed. The electronic device can detect a user operation (such as a click operation on the prompt information 706-1) acting on the prompt information 706-1. The electronic device 100 can display fitness information. For details, reference may be made to the embodiments of FIGS. 7E-7G, and details are not described herein again.

8A-8B exemplarily show UI embodiments before and after beautification of a comparison body during a photo preview.

As shown in FIGS. 8A-8B, after using the body parameters corresponding to the selected body level (such as body level 10) to perform body treatment on the body image of the person 411 collected by the 3D camera module 193, the electronic device 100 may The control 801 is displayed in the preview box 301.

As shown in FIGS. 8A-8B, when the body image of the person 411 displayed in the preview frame 301 is a body image after body treatment, the electronic device 100 can detect a user operation acting on the control 801, such as on the control 801 Press and hold the user operation without releasing. In response to this operation, the electronic device 100 can refresh the display content in the preview frame 301, and the refreshed preview frame 301 displays the body image of the person 411 actually collected by the 3D camera module 193.

As shown in FIGS. 8A-8B, when the body image of the person 411 displayed in the preview frame 301 is the body image of the person 411 actually collected by the 3D camera module 193, the electronic device 100 can detect the user acting on the control 801 An operation, such as a user operation of releasing the control 801 (that is, releasing the control 801 when it is no longer held). In response to this operation, the electronic device 100 may refresh the display content in the preview frame 301, and the body image of the person 411 displayed in the refreshed preview frame 301 is the body image after the body treatment.

In this way, the user can compare the body image of the person 411 actually collected by the 3D camera module 193 with the body image of the person 411 displayed in the refreshed preview frame 301.

9A-9C exemplarily show a UI embodiment in which a user takes a photo.

As shown in FIGS. 9A-9C, the body shape of the photographed person 411 in the preview frame 301 has a beautification compared to the actual shape of the photographed person 411. The electronic device 100 can detect a user operation (such as a click operation on the control 303) acting on the control 303, and in response to the operation, the electronic device 100 can save the image in the preview box 301, and can also display the saved in the control 304 Thumbnail of the image. Because the currently selected camera option is a portrait mode option, the image in the preview box 301 can be saved as a photo.

In addition, the electronic device 100 can also detect a user operation (such as a click operation on the control 304) acting on the control 304, and in response to the operation, the photo 801 recently saved by the electronic device 100 can be displayed. In this way, the user can view the image in the preview frame 301 that has just been saved, and the body image of the photographed person 411 included in the image is more beautified than the actual shape of the photographed person 411.

FIGS. 10A-10C exemplarily show a UI embodiment for beautifying a photographed person during video preview.

In the shooting mode list 302, the electronic device 100 can detect a user operation (such as a click operation on the recording mode option 302D) acting on the recording mode option 302D, and in response to the user operation, the electronic device 100 can turn on the recording function, and The user interface shown exemplarily in FIGS. 10A to 10C is displayed. The user interface shown in FIGS. 10A to 10C can be specifically referred to the aforementioned UI embodiment for beautifying the photographed person during the photo preview, which will not be repeated here. The difference is that at this time the control 303 can be used to monitor user operations that trigger recording. In this application, the recording mode option 302D may be referred to as the second shooting mode option.

As shown in FIGS. 10D-10F, after performing body treatment on the body image of the photographed person using the body parameters corresponding to the selected body level, the electronic device 100 can respond to the detected user operation on the control 303, The image in the preview box 301 can be started to record, and the user interface during recording can be displayed. The user interface during recording may be exemplarily shown in FIG. 10D. Then, the electronic device 100 can respond to the detected user operation on the control 10D-1, can end recording the image in the preview box 301, and can save the recorded image as a video, and display the saved in the control 304 Thumbnail of your video. The display content in the preview box 301 is updated regularly, for example, every 10 ms. During the recording process, the body image of the photographed person displayed in the preview frame 301 may be the body image of the body of the photographed person using the body parameters corresponding to the selected body level.

In addition, the electronic device 100 can also detect a user operation (such as a click operation on the control 304) acting on the control 304, and in response to the operation, the video 10F-1 recently saved by the electronic device 100 can be displayed. In this way, the user can view the video that has just been saved, and the body image of the photographed person 411 included in the video is subjected to body treatment on the body image of the photographed person using the body parameters corresponding to the selected body level.

It can be seen from the embodiments of FIGS. 10A to 10C that the user can beautify the person being photographed before starting to record. During the entire recording process, the body image of the photographed person displayed in the preview box 301 has been subjected to body treatment using the body parameters corresponding to the body level selected before starting the recording. In this way, the body shape of the photographed person's body image will be consistent throughout the video recording process.

Not limited to this, as shown in FIG. 10G, the user interface in the recording may also include a control 312 for adjusting the beauty level and/or a control 3311 for adjusting the beauty level, so that the user can also adjust The figure's body level and/or skin level.

In some embodiments, during the recording process, if it is detected that the person to be photographed has changed, the electronic device 100 may adopt the default body contour level (such as the body contour level 5) or the body contour parameter corresponding to the previously selected body contour level Take a body image of the person's body image for body treatment.

Some UI embodiments during the photo preview described in the foregoing content, such as the embodiment of undoing the beautification shown in FIGS. 5A-5D, and the embodiment of pushing fitness information shown in FIGS. 6A-6G and 7A-7G The embodiments before and after the beautification of the figure shown in FIGS. 8A-8B, and subsequent UI embodiments are also applicable to the figure beautification in the recording scene, which will not be repeated here.

FIGS. 11A to 11E exemplarily show an embodiment of a UI for beautifying a photographed person using a figure template.

As shown in FIGS. 11A-11C, the user interface 33 may further display a control 11B-1. The control 11B-1 can be used to monitor and view user operations of the open-body template. The electronic device 100 can detect a user operation (such as a click operation on the control 11B-1) acting on the control 11B-1, and in response to the operation, the user interface exemplarily shown in FIG. 11B may be displayed. The user interface may include a preview box 301, a control 303, a control 304, a control 305, a shooting mode list 302, and a figure template list 11B-2. The body template list 11B-2 may display one or more body template options 11B-3. Different figure templates can represent different figures. For the preview box 301, the control 303, the control 304, the control 305, and the shooting mode list 302, reference may be made to the description in the foregoing embodiment, and details are not repeated here.

As shown in FIGS. 11A-11C, the electronic device 100 can detect a user operation (such as a click operation on the body template option 11B-3) acting on the body template option 11B-3, which can be used to select a body template, such as Figure template "Kate". In response to this operation, the user interface exemplarily shown in FIG. 11C may be displayed. The user interface may include icons 11B-4, controls 314, a shooting mode list 302, controls 303, controls 304, and controls 305. Among them, the icon 11B-4 can be used to indicate the selected figure template. The control 314 can be used to select a beauty level. The higher the beauty level, the closer the body image of the photographed person displayed in the preview box 301 is to the body type represented by the body shape template. For the control 303, the control 304, the control 305, and the shooting mode list 302, reference may be made to the description in the foregoing embodiment, and details are not described herein again.

As shown in FIGS. 11D-11E, under the condition that the figure template has been selected, the electronic device 100 may detect the first user operation (such as a rightward sliding operation on the control 314) acting on the control 314, the first user The operation can be used to increase the figure level corresponding to the character 411 (eg, from figure level 7 to figure level 10). In response to the first user operation, the electronic device 100 can refresh the display content in the preview box 301, and the body image of the person 411 in the preview box 301 after refreshing is compared with the figure of the person 411 in the preview box 301 before refreshing The figure represented by the body image is closer to the figure represented by the selected figure template. Moreover, the higher the body contour level, the closer the body shape of the person 411 in the preview frame 301 is to the body shape represented by the selected body shape template. For example, as shown in FIG. 11E, when the body contour level is 10 (the highest body contour level), the body image represented by the body image of the person 411 in the preview frame 301 is consistent with the body shape represented by the selected body template.

As shown in FIGS. 11D-11E, under the condition that the figure template has been selected, the electronic device 100 may detect a second user operation (such as a leftward sliding operation on the control 314) acting on the control 314, the second user The operation can be used to lower the figure level corresponding to the character 411 (for example, from figure level 10 to figure level 7). In response to the second user operation, the electronic device 100 can refresh the display content in the preview box 301, and the body image of the person 411 in the preview box 301 after refreshing is compared with the figure of the person 411 in the preview box 301 before refreshing The figure represented by the body image is closer to the actual figure of the person 411.

In some embodiments, the body template options displayed in the body template list 11B-2 may be related to the gender, age, and other characteristics of the person being photographed. For example, if the person to be photographed is a female, a body template suitable for women is displayed in the body template list 11B-2, and a body template suitable for men is not displayed.

In some embodiments, in response to the detected user operation for selecting the body template, the electronic device 100 may refresh the display content in the preview box 301, and the refreshed preview box 301 represents the body image of the photographed person. The figure is the same as the figure represented by the selected figure template. That is to say, the user can directly beautify the figure of the person being photographed to be the same as the figure represented by the figure template by clicking the figure template option.

12A-12C exemplarily show a one-click UI embodiment for full-body beautification.

As shown in FIGS. 12A-12C, the user interface 33 may also display a control 12B-1. The electronic device 100 can detect a user operation (such as a click operation on the control 12B-1) acting on the control 12B-1, and in response to the operation, the user interface exemplarily shown in FIG. 12B may be displayed. The user interface may include a preview box 301, a control 303, a control 304, a control 305, a shooting mode list 302, a control 313, and an icon 12B-2. The icon 12B-2 can be used to indicate that the currently selected character beautification method is full-body beautification. Full-body beautification may refer to facial beautification and body beautification of the person being photographed. The control 313 can be used for the user to select the level of full-body beautification. For the preview box 301, the control 303, the control 304, the control 305, and the shooting mode list 302, reference may be made to the description in the foregoing embodiment, and details are not repeated here.

As shown in FIGS. 12A-12C, the electronic device 100 can detect a user operation for selecting a full-body beautification level, and in response to the operation, the electronic device 100 can beautify the body image of the person being photographed according to the selected full-body beautification level In the processing, the face image of the photographed person is subjected to skin beautification processing, and the preview frame 301 is refreshed. After refreshing, the preview box 301 displays the image of the photographed person who has undergone the body treatment and skin treatment. The higher the overall beautification level, the greater the difference between the body image of the photographed person displayed in the preview frame 301 and the actual body shape of the photographed person. The greater the difference in the actual facial skin of the person photographed.

The one-button UI embodiment for full-body beautification shown in FIGS. 12A to 12C can quickly realize the whole-body beautification of the photographed person, the operation is simple and effective, and the use efficiency of the electronic device can be improved.

13A to 13D exemplarily show a UI embodiment for beautifying multiple people.

As shown in FIGS. 13A-13B, when the preview frame 301 displays a plurality of images of the captured person captured by the 3D camera module 193, the electronic device 100 can respond to the detected user operation for selecting the body-level, using the The body parameters corresponding to the selected body level perform body processing on the body images of the plurality of photographed persons, and refresh the display content in the preview box 301. The refreshed preview frame 301 displays the body images of the plurality of photographed persons after the body treatment. That is to say, the user can beautify the plurality of photographed people with the same beauty level.

As shown in FIGS. 13C-13D, when the preview frame 301 displays a plurality of images of the captured person captured by the 3D camera module 193, the electronic device 100 can detect a user operation to select the captured person (such as when shooting Click operation on the image of the character), and a user operation to select a body level is detected. In response to this operation, the electronic device 100 can perform the body treatment on the body image of the selected photographed person using the body parameters corresponding to the selected body level, and refresh the display content in the preview box 301. The refreshed preview frame 301 displays the body image of the selected photographed person and the body image of the unselected photographed person. Among them, the body image of the selected photographed person has undergone body treatment and is not The body image of the selected person to be photographed has not undergone body treatment. That is to say, the user can select a certain person or persons from the plurality of shot persons for beautification.

In some embodiments, the electronic device 100 may recognize the image of a specific person from the images of the plurality of captured persons collected by the 3D camera module 193, such as the owner of the electronic device 100. The electronic device 100 may perform body treatment on the image of the specific person by default. Optionally, the electronic device 100 can use the image data stored on the electronic device 100 (such as photos in a gallery) to identify the image of a specific person from the images of multiple photographed persons, for example, the face image in the image of the specific person The facial images in the pictures marked, favorited or named in the gallery show the same facial features.

14A-14B exemplarily show a UI embodiment for manually performing beautification.

As shown in FIGS. 14A-14B, in the preview frame 301, the electronic device 100 can detect a user operation acting on the body image of the photographed person, as shown in FIG. 14A, a pinch gesture on the belly image. In response to the operation, the electronic device 100 may determine the partial body image selected by the operation, and then perform body treatment on the partial body image. The magnitude of the user's operation can determine the degree of body treatment. For example, the greater the magnitude, the greater the degree of body treatment.

That is to say, the user can slim down a body part through a pinch operation on the body part.

Not limited to the user operations of FIGS. 14A-14B, the stretching gesture on the leg image can be used to lengthen the leg, and the pressing operation on the tummy image can also be used to thin the belly.

15A to 15D exemplarily show a UI embodiment for beautifying a person in a captured picture.

As shown in FIG. 15A, for a picture containing a person image subjected to body treatment and/or skin treatment, the electronic device 100 may mark such a picture in the gallery, for example, displaying an indicator 15A-1 on the thumbnail of the picture . The indicator 15A-1 can be used to indicate that the image of the person in the picture has undergone body treatment and/or skin treatment.

As shown in FIGS. 15B-15D, the user interface displaying the picture 15B-1 may also display the control 15B-2. In response to the detected user operation acting on the control 15B-2, the electronic device 100 may display the control 312, the control 309, and the control 310 in the user interface displaying the picture 15B-1. For the description of the control 312, the control 309, and the control 310, reference may be made to the foregoing content, and details are not described here. In response to the detected user operation of selecting the body level (such as a swipe right operation on the control 312), the electronic device 100 can use the body parameters corresponding to the selected body level to perform the body image of the person in the picture 15B-1 Perform body treatment and refresh picture 15B-1. The body image of the person's body image in the refreshed picture 15B-1 is more beautified than the body image of the person's body image in the before-refresh picture 15B-1.

It is not limited to the foregoing pictures containing the image of the person subjected to body treatment and/or skin treatment, and the picture 15B-1 may also include pictures that have not been subjected to body treatment or skin treatment. In this way, the user can beautify the figures in the photo after the photo is taken. Similarly, the user can also beautify the figures in the captured video. For specific implementation, refer to the embodiments in FIGS. 15A to 15D, and details are not described here.

In the above UI embodiment, the image of the photographed person collected by the 3D camera module 193 displayed in the preview frame 301 may be referred to as the first image of the photographed person, and the image of the photographed person displayed in the preview frame 301 after the body treatment It can be called the second image of the person being photographed.

In the above UI embodiment, the control 312 may be referred to as a first control, and the control 801 may be referred to as a second control; the prompt information 703 in the embodiments of FIGS. 7A-7G may be referred to as first prompt information.

Based on the electronic device 100 introduced in the foregoing and the foregoing UI embodiments, the following embodiments describe the photo preview method provided by the present application. As shown in FIG. 16, the method may include:

S101. Acquire images.

Specifically, the electronic device can turn on the 3D camera module and collect color images and depth images through the 3D camera module. The color image may include an image of a person being photographed (ie, a foreground image) and a background image. The depth image may include depth information of the person being photographed.

Wherein, the color image may include a plurality of pixels, and each pixel has two-dimensional coordinates and color values. The color value can be an RGB value or a YUV value. The depth image may include a plurality of pixels, each pixel having two-dimensional coordinates and a depth value. For a position on the body of the person being photographed, the color value of the corresponding pixel in the color image of the position represents the color of the position (for example, the color of clothing, the color of bare skin, etc.), and the position corresponds to the depth image The depth value of the pixel indicates the vertical distance between the position and the electronic device (specifically a 3D camera module).

For example, as shown in FIGS. 17A-17B, for the position A (left hip point) on the body of the photographed person, the two-dimensional coordinates of the corresponding pixel point of the position A in the color image is (x1, y1), the pixel point The RGB value of is (255,255,255); the two-dimensional coordinates of the corresponding pixel in position A in the depth image are (x1,y1), and the depth value of the pixel is 350 cm. This means that the color at position A is white, and the vertical distance between position A and the electronic device is 350 cm. Similarly, for the location B (left knee point) on the body of the person being photographed, the RGB value (0,0,0) of the pixel corresponding to the location B in the color image and the pixel corresponding to the location B in the depth image A depth value of 345 cm can indicate that the color at position B is black, and the vertical distance between position B and the electronic device is 345 cm.

It can be seen that, in combination with the color image and depth information of the photographed person, the two-dimensional coordinates of each photographed part of the photographed person, the depth value relative to the 3D camera module, and the color value can be determined. Among them, the two-dimensional coordinates and depth values may represent 3D coordinates.

For example, the color image and the depth image shown in FIGS. 17A and 17B can be combined into a distribution of color values in a 3D coordinate space, as shown in FIG. 17C. The z axis represents the depth value. Among them, the 3D coordinate of position A is (x1, y1, z1), z1=350 cm, the RGB value at the 3D coordinate is (255,255,255); the 3D coordinate of position B is (x2,y2,z2), z2=345 Cm, the RGB value at this 3D coordinate is (0,0,0).

Here, the photographed part refers to the part captured by the 3D camera module. For example, when the photographed person stands facing the 3D camera module, the photographed part of the photographed person may include a face, stomach, etc. For the body part of the 3D camera module, the buttocks and back are not the part to be photographed.

S102. The electronic device may display a first user interface.

Specifically, the first user interface may be a user interface provided by the character beautification function of the "camera" application. The user interface may be the user interface shown in FIGS. 4A-4G, the user interface shown in FIGS. 5A-5D, the user interface shown in FIGS. 6A-6G, or the user interface shown in FIGS. 7A-7H. User interfaces shown in 8A-8B, user interfaces shown in FIGS. 9A-9C, user interfaces shown in FIGS. 10A-10G, user interfaces shown in FIGS. 11A-11E, shown in FIGS. 12A-12C User interface, the user interface shown in FIGS. 13A-13D.

Specifically, if the first user interface is the user interface shown in FIGS. 4A-4G, the user interface shown in FIGS. 5A-5D, the user interface shown in FIGS. 6A-6G, or the one shown in FIGS. 7A-7H User interface, the user interface shown in FIGS. 8A-8B, the user interface shown in FIGS. 9A-9C, the user interface shown in FIGS. 10A-10G, the user interface shown in FIGS. 12A-12C, and FIG. 13A- The user interface shown in FIG. 13D. Then, the body treatment method adopted by the electronic device in the subsequent S108 is the body treatment method 1. In the user interfaces shown in these figures, the body style option 310 is in a selected state, and the control 312 can be used for the user to select the body style for the person being photographed. For the specific implementation of these user interfaces, reference may be made to the corresponding aforementioned UI embodiments, which will not be repeated here.

Specifically, if the first user interface is the user interface described in the embodiments of FIGS. 11A to 11E, the body treatment method adopted by the electronic device in subsequent S108 is the body treatment method 2. For specific implementation of the user interface, reference may be made to the embodiments of FIGS. 11A to 11C, and details are not described here. At this time, the first user interface can be used for the user to select a beauty template for the photographed person, and can also be used for the user to adjust the beauty level under the selected beauty template. Regarding the user operation of the user selecting the beauty template and adjusting the beauty level, reference may be made to the embodiments of FIGS. 11A to 11E in detail, and details are not described here.

S103. Identify human bone points.

Specifically, the electronic device can use the color image of the photographed person and the human bone point positioning algorithm to identify the skeleton point of the person. Here, recognizing the bone point means identifying the 2D coordinates of the bone point. The 2D coordinates of the identified bone point and the depth value of the bone point determined in S104 can be used by the electronic device to determine the body shape of the photographed person in subsequent S106, and can also be used by the electronic device to determine the photographed person and electronic equipment in subsequent S105 The angle between the planes.

The input of the human bone point positioning algorithm (human pose estimation) may be a color image of the human body, and the output of the human bone point positioning algorithm may be the 2D coordinates of the human bone point. In this way, the electronic device can specifically take the color image of the photographed person as input and obtain the 2D coordinates of each bone point in the color image of the photographed person through the human bone point positioning algorithm. The electronic device can recognize the basic human bone points shown in FIG. 18, such as head point, neck point, left shoulder point, right shoulder point, left elbow point, right elbow point, left hand point, right hand point, left hip point, Right hip point, left knee point, right knee point, left foot point, right foot point. Not limited to that shown in FIG. 18, the electronic device can also recognize more or fewer human bone points.

Here, the human bone point positioning algorithm is obtained by training a neural network algorithm by using a large number of human color images and human bone points in the color image as training data. The reliability of the human skeleton point positioning algorithm (ie neural network algorithm) obtained by the last training is relatively high, for example, the confidence is above 90%.

S104: Determine the depth value of the bone point.

Specifically, the electronic device may use the two-dimensional coordinates of the skeleton point of the photographed person and the depth information of the photographed person identified in S102 to determine the depth value of the skeleton point of the photographed person. The determined depth value of the bone point and the 2D coordinates of the bone point determined in the foregoing S103 can be used by the electronic device to determine the body shape of the photographed person in subsequent S106, and can also be used by the electronic device to determine where the photographed person and electronic device are located in subsequent S105 The angle between the planes.

For example, as shown in FIGS. 17A-17B, in the color image of the photographed person, the two-dimensional coordinates of the left hip point (position A) of the photographed person are (x1, y1). Since the pixel at the two-dimensional coordinates (x1, y1) has a depth value of 350 cm in the color image of the person being photographed, the depth value of the left hip point is 350 cm.

In addition to the depth value of the bone point, the electronic device can also determine the depth value of other body parts of the person being photographed, such as the depth value of the belly, because the depth value of the belly can be used to determine the fatness and thinness of the belly of the person being photographed. The depth value of the belly may include the depth value of one or more feature points on the belly.

S105, perspective transformation.

When shooting, the vertical distance between each bone point of the human body and the plane where the electronic device is located is usually not equal. The more common situation is that the vertical distance between the body of the upper half of the human body (such as the head) and the plane where the electronic device is located is smaller, and the vertical distance between the body of the lower half of the human body (such as the legs) and the plane where the electronic device is located The distance is larger. This causes the person being photographed to present a visual effect of "short head and thighs" in the captured image. Optionally, the electronic device can correct the visual effect of "short head and thighs" by performing a perspective transformation on the color image before performing body treatment on the color image of the photographed person.

Specifically, the electronic device may first determine the angle between the person being photographed and the plane where the electronic device is located. Then, the electronic device can calculate the perspective transformation matrix using the included angle, and finally use the perspective transformation matrix to perform perspective transformation on the color image of the photographed person.

Assuming that the perspective transformation matrix is the following 3×3 matrix, the two-dimensional coordinates (x, y) of any pixel in the color image are transformed into the 3D coordinates (X, Y, Z) after the following matrix transformation:

In the above matrix transformation, the z value of any pixel is set to 1. The two-dimensional coordinates (x, y) can be transformed into three-dimensional coordinates (X, Y, Z) through matrix transformation. Then X, Y can be divided by the value of Z to get x’, y’. In a color image, the original two-dimensional coordinates (x, y) are converted into new two-dimensional coordinates (x', y') to obtain a perspective-transformed color image. The 2D coordinates of the human skeleton point after the perspective transformation can be obtained by multiplying the 2D coordinates of the human skeleton point before the perspective transformation by the transformation matrix.

The electronic device can determine the angle α between the person being photographed and the plane where the electronic device is located in the following two ways:

Method 1: The angle α is determined by the depth value of each bone point and the 2D coordinates.

For example, as shown in FIG. 19, the vertical distance between the left hip point P1 and the left knee point P2 of the person being photographed and the electronic device will be D1 and D2, respectively. In the color image of the person being photographed, the distance L between the left hip point P1 and the left knee point P2 can be calculated from the 2D coordinates of P1 and the 2D coordinates of P2. Therefore, the angle α between the person being photographed and the plane where the electronic device is located can be calculated as: α=arc[tan((D2-D1)/L)].

Optionally, the electronic device can calculate multiple angle values by calculating multiple sets of bone points, and then can normalize the multiple angle values. The normalized angle value can be determined as the angle α between the person being photographed and the plane where the electronic device is located.

Method 2: Collect the pitch angle θ of the electronic device through a sensor of the electronic device (such as a gyro sensor) (value range is -180° to 180°), and determine the included angle α according to the pitch angle θ. The relationship between the pitch angle θ and the included angle α may be: θ=α or θ=α-180°. Therefore, the angle α can be determined according to the pitch angle θ of the electronic device.

S106. Determine the figure of the person being photographed.

The electronic device can determine the shape of the person to be photographed based on the color image of the person to be photographed and the depth information of the person to be photographed. If the electronic device performs perspective transformation on the color image collected by the 3D camera module, the color image of the photographed person may specifically be a color image after the perspective transformation of the photographed person. The determined body shape of the photographed person can be used by the electronic device to perform a body treatment on the color image of the photographed person in S108. The electronic device determines the figure of the person being photographed, which may include one or more of the following:

1. Determine the body ratio

Specifically, the electronic device can determine the length of the bone between the bone points (such as the length of the leg, the width of the shoulder, etc.), and can determine the proportion of the body of the person to be photographed, such as the head-to-body ratio, according to the length of each bone.

Specifically, the electronic device may determine the length of the bone between the bone points according to the depth value of the bone points and the 2D coordinates of the bone points. For example, as shown in FIG. 20, the vertical distance between the left hip point P1 and the left knee point P2 of the person being photographed and the electronic device will be D1 and D2, respectively. In the color image of the photographed person, the distance L between the left hip point P1 and the left knee point P2 can be calculated from the 2D coordinates of P1 and the 2D coordinates of P2. Therefore, the bone length X of the thigh between the left hip point P1 and the left knee point P2 can be calculated as:

If the electronic device performs perspective transformation on the color image collected by the 3D camera module, the 2D coordinates of the bone points (such as the left hip point P1 and the left knee point P2) may be the 2D coordinates of the bone points after perspective transformation.

2. Determine the outline of the local body

The electronic device can determine the entire human body contour of the photographed person through the color image of the photographed person and the human body contour detection algorithm, and then can further determine the contour of the local body (such as the stomach, waist, and legs) between the bone points, that is, the local contour. The local contour of the waist can be used to determine the waist circumference, the local contour of the leg can be used to determine the leg circumference, and so on. That is to say, the contour of the partial body can be used to determine the fatness and thinness of the partial body.

The input of the human body contour detection algorithm may be a color image of the human body, and the output of the human body contour detection algorithm may be a set of two-dimensional coordinates representing the human body contour. In this way, the electronic device can specifically take the perspective-transformed color image of the photographed person as an input, and obtain a two-dimensional coordinate set of the human body contour in the color-transformed color image after the human body contour detection algorithm.

Here, the human body contour detection algorithm is obtained by training a neural network algorithm by using a large number of human body color images and a set of two-dimensional coordinates of the human body contours in the color image as training data. The reliability of the human contour detection algorithm (that is, the neural network algorithm) obtained by the final training is relatively high, for example, the confidence is above 90%.

3. Determine the depth value of the local body

Specifically, the smaller the depth value of the partial body, the closer it is to the electronic device, and thus the higher the degree of protrusion of the partial body, that is, the fatter the partial body. For example, when the person to be photographed faces the electronic device for shooting, a fat belly will protrude, and the protruding part is closer to the electronic device, that is, the depth value of the protruding part is smaller.

S107, split portrait.

Before subjecting the body image to the color image of the photographed person (S108), as shown in FIG. 21, the electronic device may divide the color image into a color image (ie, foreground image) and background image of the photographed person. Among them, (a) shows the color image of the person being photographed, and (b) shows the background image. The segmented color image of the photographed person can be used for subsequent body treatment. If the electronic device performs perspective transformation on the color image collected by the 3D camera module, the divided color image may be a color image after perspective transformation.

S108, body treatment.

Specifically, body treatment refers to the processing of the body image of the photographed person, such as image stretching, image compression, shading, etc., so that the body shape of the processed body image is compared with the actual body shape of the photographed person. beautify. If the electronic device performs perspective transformation on the color image collected by the 3D camera module, the body image of the photographed person may specifically be the body image of the photographed person after perspective transformation.

In the first user interface (such as the user interface shown in FIG. 4A ), the electronic device may perform a body treatment process when it detects a user operation for selecting a body level (that is, a user operation acting on the control 312 ). For the method of body treatment at this time, please refer to the body treatment method 1 described later.

In the first user interface (as shown in FIGS. 11A-11C), the electronic device can detect a user operation for selecting a body shape template, or select a user operation at a beauty level under the selected body shape template The body treatment is executed at the time (that is, the user operation acting on the control 312). For the method of body treatment at this time, please refer to the body treatment method 2 described in the following content.

S109, image restoration.

Specifically, the electronic device may fuse the color image and the background image of the photographed person after the body treatment. During image fusion, in order to avoid the hollow between the limb image and the background image after the body treatment, as shown in FIG. 22, the electronic device can also use the background image data at the hollow edge to interpolate the hollow part to compensate Fully hollow out the part to get the restored image. Among them, (a) in FIG. 22 shows a case where there is a hollow before the interpolation process is performed, and (b) in FIG. 22 shows a case where the hollow is repaired after the interpolation process is performed.

Finally, the electronic device can update and display the restored image in the preview box when shooting the preview. The body image of the photographed person included in the restored image is subjected to body treatment, and the body image represented by the body image is beautified compared to the actual body shape of the photographed person.

In the embodiment of the present application, the image of the photographed person (that is, the foreground image) collected by the 3D camera module may be referred to as a first image, and the image of the photographed person subjected to body treatment may be referred to as a second image.

The electronic device may adjust the contour of the first body part of the photographed person in the first image according to the first body parameters, to obtain the contour of the first body part of the photographed person in the second image. That is to say, the contour of the first body part of the photographed person in the second image can be adjusted according to the first body parameters.

The electronic device may adjust the contour of the second body part of the person in the first image according to the second body parameters to obtain the contour of the second body part of the person in the second image. That is to say, the outline of the second body part of the photographed person in the second image is adjusted according to the second body parameters.

Wherein, the first part and the second part may be different, and the first body parameter and the second body parameter may be different. That is to say, the electronic device can adjust the contours of different body parts in the first image by using different body parameters.

The following describes two implementation manners in which the electronic device performs body treatment on the body image of the photographed person.

Body treatment 1

The electronic device may perform body treatment on the body image of the photographed person according to the body treatment parameters corresponding to the selected body treatment level (the body treatment level selected by the user or the default body treatment level). For an embodiment of the UI, refer to FIG. 4A.

It is assumed that the beauty parameters corresponding to the selected beauty level can be exemplarily shown in Table 1.

body parts	Body parameters	Beautify
Legs	0.1	Elongated legs
belly	-0.05	Thin belly
Shoulder	0.3	Widen shoulders
waist	-0.1	Thin waist

Whole body (excluding head)

-0.09

Thin body

Table 1

Among them, the value of the body parameter corresponding to the leg 0.1 is a positive value, which can indicate that the leg is elongated; the value of the body parameter 0.1 can also indicate the length of the leg, and the larger the value, the leg The greater the elongation of the department. For example, 0.1 can mean stretching the leg by 10%. The example is only an implementation manner of the embodiment of the present application, and should not constitute a limitation.

Among them, the value of the body parameters corresponding to the belly -0.05 is a negative value, which can indicate thin stomach treatment; the absolute value of the value of the body parameters -0.05 can also indicate the degree of thin stomach, and the larger the absolute value, the thin stomach The greater the degree. For example, -0.05 can mean that the belly is thinned by 5%. The example is only an implementation manner of the embodiment of the present application, and should not constitute a limitation.

Among them, the value of the body-body parameter corresponding to the shoulder is 0.3, which can indicate that the shoulder is widened; the value of the body-body parameter of 0.3 can also indicate the degree of shoulder widening, and the larger the value, the greater the value The greater the width of the shoulder. For example, 0.3 may indicate a 30% widening of the shoulder. The example is only an implementation manner of the embodiment of the present application, and should not constitute a limitation.

Among them, the value of the body parameter corresponding to the waist -0.1 is a negative value, which can indicate thin waist treatment; the absolute value of the value of the body parameter -0.1 can also indicate the degree of thin waist, and the larger the absolute value, the thin waist The greater the degree. For example, -0.1 can mean thinning the waist by 10%. The example is only an implementation manner of the embodiment of the present application, and should not constitute a limitation.

Among them, the value of the body-body parameter corresponding to the whole body -0.09 is a negative value, which can indicate that the whole body is thinned; the absolute value of the body-body parameter -0.09 can also indicate the degree of weight loss. For example, 0.1 can mean 10% thinner body. The example is only an implementation manner of the embodiment of the present application, and should not constitute a limitation.

As can be seen from Table 1, the body parameters corresponding to the specific body level may be a set of body parameters, which may include the body parameters corresponding to each body part. In some embodiments, some body parameters in the set of body parameters may be invalid. For example, when the person to be photographed is identified as a pregnant woman (through a neural network algorithm), the electronic device 100 may invalidate the body parameters corresponding to the thin belly (for example, the value is set to 0), that is, the contour of the pregnant woman’s belly in the first image is not Make adjustments so that the outline of the belly of the person photographed in the second image matches the outline of the belly of the person photographed in the first image to preserve the characteristics of the pregnant woman. Table 1 only exemplarily shows an implementation of body-body parameters, which is not limited thereto. The body-body parameters may also include more or less parameters, or other different parameters.

In the present application, the body treatment refers to processing the color image of the body of the person being photographed. With reference to the body parameters exemplarily shown in Table 1, the electronic device 100 may perform one or more of the following body treatments.

A. The electronic device 100 can perform the following body treatment on the leg image of the person 411 to achieve the beautification of the elongated leg: First, in the image collected by the 3D camera module 193, according to the parameter value 0.1 corresponding to the leg and the The actual leg length of the photographed person determined in S106, and determine the respective target positions of the skeletal points of the knee point and the foot point; then stretch the leg image of the person 411, so that the leg image is stretched Skeletal points such as the back knee point and the foot point can be at their respective target positions. In this way, the legs of the person 411 represented by the leg image after the stretching process are longer than the actual legs of the person 411.

B. The electronic device 100 can perform the following body treatment on the belly image of the person 411 to realize the slimming of the belly: perform image compression processing on the belly image of the person 411, so that the belly expressed by the processed belly image is compared with the person The actual belly of 411 is thinner (or narrower); the belly image of the person 411 is shaded, so that the belly expressed by the processed belly image is flatter than the actual belly of the person 411, that is, the degree of protrusion of the belly can be reduced. Among them, the value of the body parameter corresponding to the belly -0.05 and the actual contour and depth values of the belly of the person to be photographed determined in S106 can be used to determine the range of the belly image to be compressed and the target position to be compressed, The parameter value -0.05 can also be used to determine the image range of the belly image plus shadow and the brightness of the shadow. In this way, the belly of the belly image after the processing is thinner (narrower and flatter) than the actual belly of the person 411.

C. The electronic device 100 can perform the following body treatment on the shoulder image of the person 411 to achieve the beautification of the widened shoulder: First, in the image collected by the 3D camera module 193, according to the body parameters corresponding to the widened shoulder The value 0.3 and the shoulder width determined in S106 determine the respective target positions of the left shoulder point and the right shoulder point; then the shoulder image of the person 411 is stretched so that the left shoulder point, The right shoulder point can be at its respective target position. In this way, the shoulder of the person 411 expressed by the shoulder image after the stretching process is wider than the actual shoulder of the person 411.

D. The electronic device 100 can perform the following body treatment on the waist image of the person 411 to realize the slimming of the waist: perform a compression process on the middle part of the waist image of the person 411, so that the waist represented by the image-processed waist image Thinner than the actual waist of character 411. The value of the body parameter corresponding to the waist-0.1 and the waist contour determined in S106 can be used to determine the range of the waist image to be compressed and the target position to be compressed.

E. The electronic device 100 can perform the following body treatment on the waist image of the person 411 to achieve a slim waist beautification: perform a compression process on the color image of the entire body of the person 411, in which the face image of the person 411 is excluded so that The color image of the body after the image processing is thinner than the actual figure of the person 411. Compressing the color image of the entire body of the person 411 may specifically include separately compressing the color images of each partial body between the bone points, so that the partial body represented by the color image of each partial body (such as arms and legs) , Belly, waist, etc.) are thinner than the actual parts of the character 411. Optionally, the color image (belly image) of some partial bodies can also be shaded to present a more natural slimming effect. Among them, the value of the body parameter corresponding to the whole body -0.09 and the contour of the entire body determined in S106 can be used to determine the range and target of the color image of the entire body or the color image of each partial body to be compressed position.

Not limited to some of the body treatments described in A-E above, the electronic device 100 may also perform other body treatments to achieve other body beautification.

In some embodiments, the body parameters corresponding to each body level may be equal to the base body parameters multiplied by the weight. The specific implementation may include:

In one implementation, the body parameters corresponding to the body level 10 (that is, the highest body level) may be used as the basic body parameters. The body parameters corresponding to other body levels can be obtained by multiplying the basic body parameters by the weight, and the weight is determined by the body level. The weight corresponding to the body level 10 is 1, and the weight corresponding to other body levels may be greater than 0 and less than 1. For other body-levels, the smaller the body-level, the smaller the weight. For example, suppose that Table 1 shows that the beauty level 10 corresponds to the beauty parameters, and the weight corresponding to the beauty level 9 is 0.9. Then, the body parameters corresponding to body level 9 are specifically: 0.1*0.9 (body parameters corresponding to elongated legs), -0.05*0.9 (body parameters corresponding to thin stomachs), 0.3*0.9 (body corresponding to widened shoulders) Parameter), -0.1*0.9 (body parameters corresponding to thin waist), -0.09*0.9 (body parameters corresponding to thin body).

In another implementation manner, the body parameters corresponding to the body level 1 may be used as the basic body parameters. The body parameters corresponding to other body levels can be obtained by multiplying the basic body parameters by the weight, and the weight is determined by the body level. The weight corresponding to body level 1 is 1, and the weight corresponding to other body levels may be greater than 1. For other beauty levels, the greater the body level, the greater the weight. For example, suppose that Table 1 shows that the beauty level 1 corresponds to the beauty parameters, and the beauty level 5 corresponds to the beauty weight 1.2. Then the body parameters corresponding to body level 5 are: 0.1*1.2 (body parameters corresponding to elongated legs), -0.05*1.2 (body parameters corresponding to thin belly), 0.3*1.2 (body parameters corresponding to widened shoulders) ), -0.1*1.2 (body parameters corresponding to thin waist), -0.09*1.2 (body parameters corresponding to thin body).

It is not limited to the method of multiplying the basic body parameters by the weight, and the body parameters corresponding to each body level may also be pre-configured separately, and may not have a multiple relationship of weight expression. This application does not limit how to specify the body parameters corresponding to each body level.

In some embodiments, at the selected body contour level, the body contour parameters used for body contour processing on the color image of the photographed person's body may also be related to the gender, age, and other characteristics of the photographed person. There may be some differences between the body parameters corresponding to men and the body parameters corresponding to women. Because men's body beautification needs and women's body beautification needs are usually different. For example, men prefer thinner shoulders and women prefer narrower shoulders. Then, the body parameters of widened shoulders corresponding to men may have larger values than those of women. For another example, women prefer slim waists more than men, then the absolute value of body parameters corresponding to women's thin waist may be greater than the absolute value of body parameters corresponding to men's thin waist. Optionally, there may be some differences in body parameters corresponding to people of different ages. Because the beautification needs of people of different ages can also be different. In this way, it is possible to differentiate the body of the person being photographed of different genders, ages, etc. in the preview box 301, and to achieve a better body-body imaging effect. This application does not limit how to differentiate the body parameters according to the characteristics of the person being photographed, such as gender and age.

In some embodiments, at the selected beauty level, the beauty parameters used to perform the beauty treatment on the color image of the photographed person's body may also be related to the actual figure of the photographed person. For example, the body parameters corresponding to the fat person can be obtained by multiplying the body parameters exemplarily shown in Table 1 by a weight of 1.2 to achieve the purpose of enhancing body beautification, and the body parameters corresponding to the thin person can be obtained through the body parameters exemplarily shown in Table 1. It is obtained by multiplying the weight by 0.8 to achieve the purpose of weakening the beautification. In this way, in the preview box 301, the body of the person with different body types can be differentiated to achieve a better body-effect camera effect. Not limited to the method of multiplying the weight provided by the example, the body-building parameters corresponding to people of different body types may also be pre-configured separately, and there may not be a multiple relationship of weight expression among each other. This application does not limit how to define body parameters corresponding to people of different body types.

Body treatment 2

The electronic device may use the selected body shape template (the body shape template selected by the user or the default body shape template) to perform beauty treatment on the color image of the photographed person. For UI embodiments, reference may be made to FIGS. 11A-11E.

In method 2, the posture of the figure template and the posture of the subject may be different. The posture of the photographed object can be determined according to the color image of the photographed person and the depth information of the photographed person. At this time, the electronic device can transform the posture of the figure template into the posture of the photographed person through similar transformation. Specifically, the electronic device can compare the displacement of the bone points of the two poses in the two-dimensional space, and the relative angle between the two limbs connected by the bone points of the photographed person. Then, the electronic device can rotate or translate the bone points of the body template and the limbs connected to the bone points, so that the posture of the transformed body template is consistent with the posture of the object to be photographed.

After adjusting the posture of the body template, the electronic device can align the bone points of the person being photographed with the bone points of the body template, and then pull the image of the body of the person being photographed according to the difference between the body template and the contour of the body of the person being photographed Stretching treatment (such as elongated legs) or compression treatment (such as thin legs). The degree of stretch or compression can be determined by the body level under the selected body template. The higher the body level, the greater the degree of stretch or compression. For the selection of the beauty level, reference may be made to the embodiments in FIGS. 11D-11E. For example, as shown in FIG. 11E, when the body level is 10 (the highest body level), the image of the person’s limb can be stretched (such as elongated legs) or compressed (such as skinny legs) so that The contour of the limb represented by the image of the stretched or compressed limb is consistent with the contour of the limb of the body template.

In some embodiments, after acquiring color images and depth information through the 3D camera module, the electronic device may perform image preprocessing. The image pre-processing may include pre-processing operations such as image enhancement, depth complementation, and denoising of color images and depth information. Among them, the image enhancement may be to enhance the brightness of the image for a dark or low-light environment. Depth completion may refer to smoothing the holes in the depth map. Denoising can refer to removing some noise in the depth map to avoid the noise above the depth value from affecting the calculation of the bone point depth.

In some possible cases, the electronic device detects in S105 that the angle between the person being photographed and the plane on which the electronic device is located is too large and exceeds a preset angle threshold (such as 65°). In response to this situation, the electronic device may not perform perspective transformation on the color image collected by the 3D camera, and may not perform body treatment on the color image of the photographed person. In addition, the electronic device can also output prompt information in the preview box 301 to remind the user that the body treatment is not supported in this case.

The photo preview method described in the embodiment of FIG. 16 can realize the adjustment of the body image of the photographed person displayed in the preview frame during shooting preview, so that the body shape of the adjusted body image occurs compared with the actual body shape of the photographed person Beautify the shape, the user's operation is simple and intuitive, which can effectively improve the utilization rate of electronic equipment.

It can be understood that, for the user operation and the corresponding response detected by the electronic device in the first user interface, reference may be made to the foregoing UI embodiment, and details are not described herein again. For content not mentioned in the method embodiment of FIG. 16, reference may be made to the foregoing UI embodiment, and details are not described herein again.

FIG. 23 shows a structural diagram of a functional module included in an electronic device in conjunction with a 3D camera module 193. Expand below:

The 3D camera module 193 can be used to collect color images and depth information. The color image may include the color image of the person to be photographed and the color image of the background. The depth information may include depth information of the person being photographed and depth information of the background. The color image of the photographed person may include a body image and a face image. In a specific implementation, the 3D camera module 193 may be composed of a color camera module and a depth camera module. The depth camera module may be a TOF depth camera module or a structured light camera module.

The human skeleton point positioning algorithm module can be used to identify the skeleton point of the photographed person by using the color image of the photographed person and the human skeleton point positioning algorithm. For specific implementation of the human bone point positioning algorithm module, reference may be made to S103 in the method embodiment of FIG. 16, and details are not described herein again.

The skeletal point depth acquisition module can be used to determine the skeletal point of the photographed person based on the 2D coordinates of the skeletal point of the photographed person identified by the human skeletal point positioning algorithm module and the depth information of the photographed person collected by the 3D camera module 193 Depth value. For specific implementation of the bone point depth acquisition module, reference may be made to S104 in the method embodiment of FIG. 16, and details are not described herein again.

The bone length calculation module can be used to determine the length of the bone between the bone points according to the depth value of each bone point determined by the bone point depth acquisition module and the 2D coordinates of the bone point of the photographed person identified by the human bone point positioning algorithm module. For specific implementation of the bone length calculation module, reference may be made to S106 in the method embodiment of FIG. 16, and details are not described here.

The perspective transformation module can be used to determine the depth value of each bone point determined by the bone point depth acquisition module and the length of the bone determined by the bone length calculation module, to determine the angle α between the photographed person and the plane where the electronic device is located, and use this angle to calculate The perspective transformation matrix is obtained, and finally the perspective transformation matrix is used to perform perspective transformation on the color image of the photographed person. Optionally, the perspective transformation module may also determine the angle α according to the pitch angle of the electronic device. For specific implementation of the perspective transformation module, reference may be made to S105 in the method embodiment of FIG. 16, and details are not described herein again.

The portrait segmentation module can be used to segment the color image obtained by the perspective transformation module into a color image (ie, a foreground image) and a background image of the person being photographed. For the specific implementation of the portrait segmentation module, reference may be made to S107 in the method embodiment of FIG. 16, and details are not described here.

The body processing module can be used to process the body image of the person after the perspective transformation, such as image stretching, image compression, shadowing, etc., so that the body shape of the processed body image occurs compared to the actual body shape of the person To beautify the body. In one implementation, the body treatment module may be used to perform body treatment on the body image of the photographed person according to the body treatment parameters corresponding to the selected body treatment level (user-selected body treatment level or default body treatment level). The beauty parameters can be determined by the beauty parameter determination module. In another implementation manner, the body treatment module may be used to perform body treatment on the color image of the photographed person using the selected body template (user-selected body template or default body template). For a specific implementation of the body treatment module, reference may be made to S108 in the method embodiment of FIG. 16, and details are not described herein again.

The image restoration module can be used to fuse the color image and background image of the photographed person after body treatment. During image fusion, in order to avoid the hollow between the limb image and the background image after the body treatment, the image repair module can also be used to interpolate the hollow part using the background image data at the hollow edge to complete the hollow part, To get the repaired image.

The image finally output by the image repair module can be updated and displayed in the preview box of the first user interface.

The above is only the specific implementation of this application, but the scope of protection of this application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. It should be covered by the scope of protection of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (17)

1. An electronic device photo preview method, characterized in that the electronic device has a 3D camera module, the method includes:

   Turn on the 3D camera module;

   Collecting the first image of the person being photographed through the 3D camera module;

   Displaying a first graphical user interface, where the first graphical user interface includes a preview box, and the preview box displays a first image of the photographed person;

   A first user operation is detected, and in response to the first user operation, a second image of the photographed person is displayed in the preview box of the first graphical user interface; The contour of the first body part of the photographed person is adjusted according to the first body parameters, and the contour of the second body part of the photographed person in the second image is adjusted according to the second body parameters. One part is different from the second part, and the first body parameters are different from the second body parameters.
2. The method according to claim 1, wherein the first graphical user interface further comprises a first control, and the first user operation is a user operation for selecting a body level that acts on the first control; The higher the beauty level selected by the first user operation, the greater the gap between the outline of the first part in the second image and the outline of the first part in the first image, in the second image The greater the difference between the outline of the second part and the outline of the second part in the first image.
3. The method of claim 2, wherein the first user operation comprises: a second user operation acting on the first control, and the second user operation is used to increase a body level;

   The displaying the first image of the photographed person in the preview box of the first graphical user interface in response to the first user operation specifically includes:

   In response to the second user operation, refresh the display content in the preview box; the outline of the first part in the second image in the preview box after refreshing is compared with the preview box before refreshing The outline of the first part in the second image is larger than the outline of the first part in the first image; the second image in the preview frame after the refresh The contour of the second part is larger than the contour of the second part in the second image in the preview frame before refreshing, and the gap between the contour of the second part in the first image is greater.
4. The method according to any one of claims 2-3, wherein the first user operation comprises: a third user operation acting on the first control, the third user operation is used to reduce the body level;

   The displaying the first image of the photographed person in the preview box of the first graphical user interface in response to the first user operation specifically includes:

   In response to the third user operation, refresh the display content in the preview box; the outline of the first part in the second image in the preview box after refreshing is compared with the preview box before refreshing The outline of the first part in the second image is smaller than the outline of the first part in the first image; the second image in the preview frame after the refresh The contour of the second part is smaller than the contour of the second part in the second image in the preview frame before refreshing, and the gap between the contour of the second part in the first image is smaller.
5. The method according to any one of claims 1 to 4, further comprising: when it is detected that the photographed person is a pregnant woman, displaying prompt information in the preview box, the prompt information is used for It is suggested that the photographed person is a pregnant woman; the contour of the stomach of the photographed person in the second image is consistent with the contour of the stomach of the photographed person in the first image.
6. The method according to any one of claims 1-5, further comprising: in response to the detected fourth user operation, the color of the face skin of the photographed person in the second image The value is adjusted.
7. The method according to any one of claims 1-6, further comprising:

   The electronic device displays a second control in the preview box;

   Displaying the first image in the preview box in response to the detected user operation of pressing and holding on the second control without releasing it;

   In response to the detected user operation of releasing the second control, the second image is displayed in the preview box.
8. The method according to any one of claims 1-7, further comprising:

   Displaying first prompt information in the preview box, the first prompt information is used to prompt the difference between the contour of the first part in the second image and the contour of the first part in the first image 2. The difference between the outline of the second part in the second image and the outline of the second part in the first image.
9. The method according to any one of claims 1-8, wherein the first graphical user interface further comprises: a shooting control; the method further comprises: in response to a detected action on the shooting control The user operates to save the second image displayed in the preview frame.
10. The method of claim 9, wherein the first graphical user interface further comprises: a shooting mode list, the shooting mode list includes a plurality of shooting mode options; the plurality of shooting modes includes: a first shooting Mode options and second shooting mode options;

    When the first shooting mode option is in a selected state, the shooting control is specifically used to monitor a user operation that triggers a photograph, and the user operation that acts on the shooting control in response to the detection is saved in the preview The second image displayed in the frame specifically includes: in response to a detected user operation acting on the shooting control, saving the second image displayed in the preview frame as a photo;

    When the second shooting mode option is in a selected state, the shooting control is specifically used to monitor a user operation that triggers recording, and the response to the detected user operation acting on the shooting control is to save the preview The second image displayed in the frame specifically includes: in response to the detected user operation acting on the shooting control, saving the second image displayed in the preview frame as a video.
11. The method according to any one of claims 1-10, wherein, in response to the first user operation, the image of the person being photographed is displayed in a preview box of the first graphical user interface Before the second image, it further includes: identifying a human skeleton point based on the image of the photographed person.
12. The method according to any one of claims 1 to 11, wherein in response to the first user operation, the image of the photographed person is displayed in a preview box of the first graphical user interface Before the second image, the method further includes: determining an angle between the photographed person and the plane where the electronic device is located, and the angle does not exceed a preset angle threshold.
13. The method according to claim 12, wherein before the second image of the photographed person is displayed in the preview box of the first graphical user interface in response to the first user operation, include:

    Determine a perspective transformation matrix according to the angle between the photographed person and the plane where the electronic device is located;

    Perform perspective transformation on the first image of the photographed person according to the perspective transformation matrix.
14. An electronic device includes a 3D camera module, a display screen, a touch sensor, a memory, one or more processors, multiple application programs, and one or more programs; wherein the one or more programs are stored in the The memory; characterized in that, when the one or more processors execute the one or more programs, the electronic device enables the electronic device to implement the method according to any one of claims 1 to 13.
15. A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, when the processor executes the computer program, the computer device is realized The method according to any one of claims 1 to 13.
16. A computer program product containing instructions, characterized in that, when the computer program product runs on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 13.
17. A computer-readable storage medium including instructions, characterized in that when the instructions run on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 13.

Images