WO2019071562A1

WO2019071562A1 - Data processing method and terminal

Info

Publication number: WO2019071562A1
Application number: PCT/CN2017/106014
Authority: WO
Inventors: 吴清亮; 陈绍君; 陈晓晨
Original assignee: 华为技术有限公司
Priority date: 2017-10-13
Filing date: 2017-10-13
Publication date: 2019-04-18
Also published as: CN110168599B; CN110168599A

Abstract

Disclosed by the embodiments of the present application is a data processing method, used to enhance the image capturing experience of a user and comprising: a terminal obtaining first point cloud information of a standard portrait, the first point cloud information being used to indicate coordinates of the standard portrait on a display screen of a terminal; the terminal obtaining second point cloud information of a sticker, the second point cloud information being at least part of the first point cloud information, the second point cloud information being used to indicate a location of the sticker relative to the standard portrait, the second point cloud information being artificially formulated, and the sticker being a pattern that may be displayed in the display screen; when the terminal detects a portrait in an image capturing preview state, the terminal generating third point cloud information, the third point cloud information being used to indicate the coordinates of the portrait on the display screen of the terminal in the image capturing preview state; when the third point cloud information matches the first point cloud information and the user selects the sticker, the terminal displaying the sticker according to the second point cloud information. Also provided by the embodiments of the present application is a terminal, which is used for improving the image capturing experience of a user.

Description

Data processing method and terminal

Technical field

The present application relates to the field of terminal technologies, and in particular, to a data processing method and a terminal.

Background technique

Nowadays, with the continuous advancement of technology, terminals with shooting functions are becoming more and more popular. In order to meet the individual needs of shooting and to enhance the fun of shooting, users can select personalized stickers during the shooting process and finally shoot them. A photo of a face and a sticker.

In the prior art, after the picture design of the sticker is completed, the coordinates are set based on the display resolution of the terminal to arbitrarily define the initial display position of the sticker in the terminal. When the terminal detects the face during the shooting process, the user can select the sticker, and then the user can select the sticker. The terminal combines the sticker with the face. For example, the sticker is a pair of rabbit ears. According to the sticker, the initial display position of the terminal is near the mouth of the face, and then the terminal adjusts the rabbit ear to the top position of the face to complete the stitching.

However, the initial display position of the arbitrarily defined sticker in the terminal is often far from the actual display position of the sticker in the actual scene relative to the face, so that the terminal often needs to spend more time to adjust the position of the sticker, resulting in poor user experience. .

Summary of the invention

The embodiment of the present application provides a data processing method and a terminal, which are used to improve user experience.

In view of this, the first aspect of the embodiments of the present application provides a data processing method, including:

The terminal acquires the first point cloud information of the standard portrait, and the point cloud information refers to a set of vectors in a three-dimensional coordinate system. These vectors are usually expressed in the form of X, Y, Z three-dimensional coordinates, and are generally used to represent the outer surface shape of an object. It can be understood that the first point cloud information can indicate the shape and display of the standard portrait displayed on the terminal. position;

The terminal can also obtain the second point cloud information of the sticker based on the obtained first point cloud information. The sticker refers to the picture that the user can add to the terminal interface when using the shooting function, and the second point cloud information is the first point. At least a part of the information in the cloud information, it can be understood that the second point cloud information can indicate the position of the sticker on the terminal relative to the standard portrait. In addition, the second point cloud information is artificially formulated according to the actual content of the sticker, for example, The sticker content is a pair of glasses, then the sticker is designed near the eyes of the standard portrait. If the sticker content is a hat, the sticker is designed near the top of the standard portrait;

Further, when the terminal detects the portrait in the shooting preview state, the terminal generates third point cloud information, and the third point cloud information may indicate the shape and the displayed position of the portrait displayed by the terminal in the shooting preview state in real time. It can be understood that the third point cloud information is not fixed like the first point cloud information, and the third point cloud information will change according to the change of the portrait in the lens.

Next, when the third point cloud information matches the first point cloud information and the user selects the sticker, the terminal displays the sticker according to the second point cloud information. It can be understood that the terminal can determine whether the portrait is currently detected according to a certain rule. , that is, it will judge whether the third point cloud information matches the first point cloud information, and if it matches, the display shape of the current portrait is displayed. And the display position is similar to the standard portrait, the terminal can recognize that the portrait is detected, so the sticker selected by the user can be displayed on the terminal according to the second point cloud information.

In the embodiment of the present application, the terminal may obtain the first point cloud information of the standard portrait and the second point cloud information of the sticker, and the second point cloud information is used to indicate the position of the sticker relative to the standard portrait, when the terminal detects the portrait The third point cloud information of the portrait may be generated. Further, if the third point cloud information matches the first point cloud information and the user selects the sticker, the terminal displays the sticker according to the second point cloud information, because the second point cloud of the sticker The information is artificially based on the content of the sticker. For example, the sticker content is a pair of glasses, then the sticker is near the eye of the standard portrait. If the sticker content is a hat, the sticker is near the top of the standard portrait, so when When the third point cloud information matches the first point cloud information, the second point cloud information of the sticker is that the initial display position of the sticker and the actual display position of the sticker in the actual scene relative to the portrait are smaller. The time for the subsequent terminal to adjust the position of the sticker is saved, and the user experience is improved.

With reference to the first aspect of the embodiments of the present application, in a first implementation manner of the first aspect of the embodiments, the method further includes:

The terminal selects a target SDK from a set of software development kits (SDKs), and all SDKs in the SDK set have a unified SDK interface. It can be understood that the SDKs in the SDK set include third-party development. The camera stitching algorithm, the portrait beauty algorithm, the filter shooting algorithm and the portrait detection algorithm and the like, the camera APP can implement the above various camera functions by calling a third-party SDK.

With reference to the first implementation manner of the first aspect of the embodiment of the present application, in the second implementation manner of the first aspect of the application, the terminal selects the target SDK from the SDK set, including:

The terminal can select the target SDK from the integrated SDK set by means of intelligent selection. Specifically, each SDK has a corresponding SDK parameter, which includes parameters of the user skin level, parameters of the preview algorithm processing rate, and stickers. Following the parameters of the rate, the three parameters can be brought into the following formula to calculate the calculation result, and the calculation result with the largest value is selected from the set of calculation results, and the SDK corresponding to the calculation result is determined as the target SDK.

With reference to the second embodiment of the first aspect of the embodiment of the present application, in the third real-time mode of the first aspect of the embodiment of the present application, the formula includes:

R=α*B+γ*P+β*F;

Where R is the calculation result, B is the parameter of the user's skin level, P is the parameter of the preview algorithm processing rate, F is the parameter of the sticker followability rate, and α, β, and γ are preset weight coefficients. .

It should be noted that the preset weight coefficient can be different in different shooting modes. For example, in the camera mode, the requirement for skin color is higher, then the value of α can be adjusted correspondingly, and the sticker followability rate requirement in the recording mode is required. If the value is higher, the value of β can be adjusted accordingly. The values of α, β, and γ are not limited herein.

With reference to any one of the first embodiment of the first aspect of the embodiment of the present application to the third embodiment of the first aspect of the present application, the fourth aspect of the first aspect of the embodiment of the present application In an embodiment, the unified SDK interface definition includes:

Download and register stickers, capture interface initialization, perform sticker previews, process shooting data, and end the process.

With reference to the first aspect of the embodiments of the present application, in a fifth implementation manner of the first aspect of the embodiments, the method further includes:

The terminal selects the target shooting algorithm from the set of shooting algorithms. It can be understood that the terminal can also integrate multiple shooting algorithms, that is, the developer of the shooting algorithm discloses the shooting algorithm. In this scenario, the SDK is no longer needed. By directly selecting a certain shooting algorithm, various shooting functions such as portrait beauty, portrait detection, filter shooting, and sticker stitching can be realized. The selection of the target shooting algorithm can be selected by the terminal intelligently, or can be selected by the user according to preferences. This is not limited here.

With reference to the fifth implementation manner of the first aspect of the embodiment of the present application, in the sixth implementation manner of the first aspect of the application, the target shooting algorithm includes:

Portrait beauty algorithm, portrait detection algorithm, filter shooting algorithm and/or sticker flattening algorithm.

A second aspect of the embodiment of the present application provides a terminal, including:

a first acquiring unit, configured to acquire first point cloud information of the standard portrait, where the first point cloud information is used to indicate coordinates of the standard portrait on the display screen of the terminal;

a second acquiring unit, configured to acquire second point cloud information of the sticker, where the second point cloud information is at least a part of information in the first point cloud information, and the second point cloud information is used to indicate a position of the sticker relative to the standard portrait, The two-point cloud information is artificially formulated, and the sticker is a pattern that can be displayed in the display screen;

a generating unit, configured to generate a third point cloud information when the terminal detects the portrait in the shooting preview state, where the third point cloud information is used to indicate the coordinates of the portrait on the display screen in the shooting preview state;

And a display unit, configured to display the sticker according to the second point cloud information when the third point cloud information matches the first point cloud information and the user selects the sticker.

With reference to the second aspect of the embodiments of the present application, in the first implementation manner of the second aspect of the embodiment, the terminal further includes:

The first selection unit is configured to select a target SDK from the SDK set, and all the SDKs in the SDK set have a unified SDK interface, and the target SDK is used to implement the flattening of the portrait and the sticker.

With reference to the first implementation manner of the second aspect of the embodiment of the present application, in the second implementation manner of the second aspect of the embodiment of the present application, the first selecting unit includes:

An obtaining module, configured to obtain an SDK parameter set corresponding to the SDK set, where the SDK parameter set includes a parameter set of the user skin level, a parameter set of a preview algorithm processing rate, and a parameter set of a sticker followability rate;

a calculation module, configured to substitute the SDK parameter set into a formula to obtain a calculation result set;

a first determining module, configured to determine a target calculation result from the set of calculation results, where the target calculation result is a calculation result with the largest value in the calculation result set;

The second determining module is configured to determine a target SDK corresponding to the target calculation result.

With reference to the second aspect of the embodiments of the present application, in a third implementation manner of the second aspect of the embodiments, the terminal further includes:

And a second selecting unit, configured to select a target shooting algorithm from the set of shooting algorithms, where the target shooting algorithm is used to implement the stitching of the portrait and the sticker.

A third aspect of the embodiment of the present application provides a terminal, including:

a processor, a memory, a bus, and an input and output interface;

A program code is stored in the memory;

When the processor calls the program code in the memory, the following operations are performed:

Obtaining the first point cloud information of the standard portrait, the first point cloud information is used to indicate the coordinates of the standard portrait on the display screen of the terminal;

Obtaining a second point cloud information of the sticker, the second point cloud information is at least part of the information in the first point cloud information, the second point cloud information is used to indicate the position of the sticker relative to the standard portrait, and the second point cloud information is artificially drawn The sticker is a pattern that can be displayed in the display screen;

When the terminal detects the portrait in the shooting preview state, the third point cloud information is generated, and the third point cloud information is used to indicate the coordinates of the portrait on the display screen when the terminal is in the preview state;

When the third point cloud information matches the first point cloud information and the user selects the sticker, the sticker is displayed according to the second point cloud information.

A fourth aspect of the embodiments of the present application provides a computer readable storage medium having instructions stored in a computer to cause a computer to execute the flow in the data processing method of the first aspect described above when running on a computer.

A fifth aspect of the embodiments of the present application provides a computer program product, which, when run on a computer, causes the computer to execute the flow in the data processing method of the first aspect described above.

As can be seen from the above technical solutions, the embodiments of the present application have the following advantages:

In the technical solution provided by the embodiment of the present application, the terminal may obtain the first point cloud information of the standard portrait and the second point cloud information of the sticker, and the second point cloud information is used to indicate the position of the sticker relative to the standard portrait, when the terminal When the portrait is detected, the third point cloud information of the portrait may be generated. Further, if the third point cloud information matches the first point cloud information and the user selects the sticker, the terminal displays the sticker according to the second point cloud information, due to the sticker The second point cloud information is artificially based on the content of the sticker. For example, the sticker content is a pair of glasses, then the sticker is near the eye of the standard portrait. If the sticker content is a hat, the sticker is on the top of the standard portrait. Nearby, so when the third point cloud information matches the first point cloud information, the second point cloud information of the sticker is the initial display position of the sticker and the actual display position of the sticker relative to the portrait in the actual scene. The error is small, which saves the time for the subsequent terminal to adjust the position of the sticker, thereby improving the user experience.

DRAWINGS

FIG. 1 is a schematic diagram of an application scenario of a data processing method according to an embodiment of the present application;

2 is a schematic diagram of an embodiment of a data processing method in an embodiment of the present application;

3(a) is a schematic diagram showing a standard portrait displayed on a terminal in the embodiment of the present application;

FIG. 3(b) is a schematic diagram showing a sticker displayed on a terminal in an embodiment of the present application;

FIG. 3(c) is a schematic diagram of a scene applied to a portrait and a sticker in the embodiment of the present application;

FIG. 3( d ) is a schematic diagram of another scene applied to the combination of a portrait and a sticker in the embodiment of the present application;

FIG. 3( e ) is a schematic diagram of another scene applied to a portrait and a sticker in the embodiment of the present application;

FIG. 3(f) is a schematic diagram of another scene applied to a portrait and a sticker in the embodiment of the present application;

4 is a schematic diagram of another embodiment of a data processing method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of another application scenario of a data processing method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a scenario applied to a user operating terminal to download a sticker according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an embodiment of a terminal in an embodiment of the present application;

FIG. 8 is a schematic diagram of another embodiment of a terminal according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a terminal in an embodiment of the present application.

Detailed ways

The embodiment of the present application provides a data processing method and a terminal, between the initial display position of the sticker and the final actual display position when the user selects the sticker by pre-defining the initial display position of the sticker in the terminal. The error is small, which saves the time for the subsequent terminal to adjust the position of the sticker and improves the user experience.

The terms "first", "second", "third", "fourth", etc. (if present) in the specification and claims of the present application and the above figures are used to distinguish similar objects without having to use To describe a specific order or order. It is to be understood that the data so used may be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than what is illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

The embodiment can be applied to the application scenario shown in FIG. 1. The terminal can communicate with a third-party server, and the third-party server stores a sticker material designed by a third party, and the camera application (application, APP) in the terminal starts. The third-party SDK integrated in the camera APP can download the third-party sticker material from the third-party server and register the downloaded third-party sticker material into the sticker management module of the camera APP for the user to select, in addition, the camera will The captured image data is sent to the camera APP, combined with the sticker material selected by the user, and the third party SDK performs portrait recognition and the sticker and the portrait are combined to finally realize the shooting preview and the image saving.

It can be understood that, in this embodiment, the third-party SDK includes a third-party developed sticker flattening algorithm, a portrait beauty algorithm, a filter shooting algorithm, and a portrait detection algorithm, and the camera APP can be implemented by calling a third-party SDK. A variety of camera functions above.

It should be noted that the camera APP is executed in the application processor of the terminal, and the application processor can perform data interaction with the camera through its external interface. It can be understood that the image data format output by the camera may be a YUV format, or may be Other formats, such as the RGB format, are not limited herein.

In addition, the terminal in the embodiment of the present application may be a mobile phone, a tablet computer, a wearable device, an augmented reality (AR), a virtual reality (VR) device, a notebook computer, a super mobile personal computer (ultra-mobile). The personal computer (UMPC), the netbook, the personal digital assistant (PDA), and the like have a shooting function, and the embodiment of the present application does not impose any limitation.

The following is a detailed description of the data processing method in the embodiment of the present application. Referring to FIG. 2, an embodiment of the data processing method in the embodiment of the present application includes:

201. The terminal acquires the first point cloud information of the standard portrait.

In this embodiment, the terminal may acquire the first point cloud information of the standard portrait, where the point cloud information refers to a set of vectors in a three-dimensional coordinate system. These vectors are usually represented in the form of X, Y, Z three-dimensional coordinates, and are generally used primarily to represent the outer surface shape of an object. Most point cloud information is generated by 3D scanning devices, such as lidar (2D/3D), stereo camera, time-of-flight camera, These devices measure information on a large number of points on the surface of the object in an automated manner, and then output point cloud information using a data file.

The standard portrait is an artificially defined 3D portrait model. It can be understood that the first point cloud information of the standard portrait can be used to reflect the outer surface shape of the standard portrait and its display position at the terminal. The standard portrait can refer to the standard. The model of the human head can also refer to a model including a standard human head and a standard human torso, which is not limited herein. This embodiment describes the standard portrait as a standard human head, as shown in FIG. 3(a), and the terminal is a mobile phone. For example, the figure shows the display position and display shape of the standard portrait based on the first point cloud information on the terminal interface.

It can be understood that FIG. 3( a ) is only an example. In actual application, the standard portrait does not have to be displayed on the interface of the terminal according to the first point cloud information, and the first point cloud information of the standard portrait is in three-dimensional coordinates. The form can be expressed as shown in Table 1 below. Table 1 below is a part of the three-dimensional coordinates of the first point cloud information of the standard portrait. In actual application, the number and value of the specific coordinates in the table may vary. Make a limit.

Table 1

X坐标X coordinate	Y坐标Y coordinate	Z坐标Z coordinate
-3.560004-3.560004	58.36919458.369194	0.2845710.284571
-2.719511-2.719511	58.20719558.207195	1.0344741.034474
-1.569996-1.569996	58.04937058.049370	1.7265451.726545
-4.797817-4.797817	58.37485558.374855	-0.640903-0.640903
-0.730695-0.730695	57.88474757.884747	2.0665512.066551
-0.000000-0.000000	57.82104557.821045	2.1392352.139235
-0.658006-0.658006	57.52378557.523785	2.2902492.290249
-1.431864-1.431864	57.71991057.719910	2.0496032.049603
0.0002200.000220	57.45122957.451229	2.3339962.333996
-2.444017-2.444017	60.43586760.435867	0.1357460.135746
-2.659698-2.659698	60.94844160.948441	-0.729706-0.729706
-4.007832-4.007832	58.66256358.662563	-0.737348-0.737348
…...	…...	…...
0.0000000.000000	66.14638566.146385	4.6425034.642503

202. The terminal acquires a second point cloud information of the sticker.

In this embodiment, the terminal acquires the second point cloud information of the sticker, where the second point cloud information is at least a part of the information of the first point cloud information. It can be understood that after the picture design of the sticker is completed, the first point is needed. The cloud information is selected as the second point cloud information, that is, the position of the sticker relative to the standard portrait is defined. As shown in FIG. 3(b), the sticker picture is a pair of sunglasses, then the first point can be selected. The three-dimensional coordinates in the vicinity of the standard portrait eyes in the cloud information are used as the second point cloud information, and the figure is the display position of the sticker according to the second point cloud information at the terminal.

It can be understood that the second point cloud information of the sticker is artificially formulated according to the actual content of the sticker. For example, if the sticker is a hat, then the three-dimensional coordinates near the top of the standard portrait image in the first point cloud information can be selected as the first Two points of cloud information, so in actual applications, different stickers can correspond to the same point cloud information, and can also correspond to different point cloud information, which is not limited herein.

It should be noted that a complete set of sticker material includes a picture of the sticker and a second point cloud information of the sticker, wherein the image format of the sticker may be a bitmap file (bitmap, bmp), or other formats. For example, it may be a portable network image (png), which is not limited herein.

203. When the terminal detects the portrait in the shooting preview state, the terminal generates third point cloud information.

In this embodiment, the user opens the camera APP to enter the shooting preview state, the camera can acquire image information, and display a preview image on the terminal display screen, and the terminal can detect the image acquired by the camera in real time, and the portrait appears in the shooting range of the camera. The terminal generates the third point cloud information of the portrait. It can be understood that the third point cloud information is also represented in the form of three-dimensional coordinates, and is used to indicate that the portrait currently acquired by the camera is in the state of the preview preview. The actual display shape and the actual display position of the terminal, the third point cloud information is generated according to the portrait detection algorithm of the third party SDK, and the third point cloud information changes in real time as the portrait image acquired by the camera moves.

204. When the third point cloud information matches the first point cloud information and the user selects the sticker, the terminal displays the sticker according to the second point cloud information.

In this embodiment, when the third point cloud information matches the first point cloud information, the user may select a sticker on the interface of the terminal, and the terminal displays the sticker according to the second point cloud information, so that the preview is simultaneously displayed on the display screen of the terminal. Portraits and stickers, to achieve the combination of portraits and stickers, it can be understood that the display of stickers is based on the third-party SDK's sticker flattening algorithm, as shown in Figure 3 (c), the terminal determines the third point cloud information and When a little cloud information matches, the user can select a sticker from the list of sticker materials on the terminal interface, and the terminal will display the sticker on the terminal interface according to the second point cloud information of the sticker. It can be understood that in practical applications The third point cloud information of the actual portrait is not completely coincident with the first point cloud information of the standard portrait. Therefore, according to the different situations of the portrait on the terminal display screen, the terminal will scale or shift the sticker by the sticker flatning algorithm. The stitching of the sticker and the portrait is realized, and the display scenes of the specific portrait and the sticker are separately described below.

It should be noted that the terminal may determine, according to the coincidence ratio of the third point cloud information and the first point cloud information, whether the third point cloud information matches the first point cloud information, specifically, the terminal compares the third point cloud information with the third point. The set of coordinates and the three-dimensional coordinate set of the first point cloud information, and set a threshold value, for example, 90%, when the third point cloud information and the first point cloud information coincide in the three-dimensional coordinate number reaches the first point cloud information in three-dimensional When the total number of coordinates is 90%, the terminal determines that the third point cloud information matches the first point cloud information. It can be understood that the threshold value may be other values, for example, 95%, which is not limited herein.

It should be noted that, based on the manner of determining whether the third point cloud information matches the first point cloud information, there are also multiple scene terminals that can also determine that the third point cloud information matches the first point cloud information, and are respectively described below. :

Optionally, as shown in FIG. 3(d), when the distance between the camera and the portrait is extended, the shape displayed by the portrait on the terminal interface is proportionally reduced, and the third point cloud information may not be the same. A little cloud information matches, but if the current third point cloud information matches the point cloud information obtained by scaling the first point cloud information by a certain degree, the terminal may also determine that the third point cloud information matches the first point cloud information, Specifically, the scaling may be 30% or 20%, which is not limited herein. It can be understood that the second point cloud information according to the sticker flattening algorithm sticker is also scaled according to the first point cloud information. To zoom, the terminal will display the sticker according to the point cloud information obtained by scaling the second point cloud information to realize the flattening of the sticker and the portrait. In addition, when the camera and the portrait are close, the description is the same as the above description, and details are not described herein. .

Optionally, as shown in FIG. 3(e), the position of the portrait captured by the terminal has a significant displacement change relative to the position of the standard portrait, if the current third point cloud information and the first point cloud information are changed by corresponding displacements. When the point cloud information is matched or the third point cloud information is matched with the point cloud information obtained after the first point cloud information is scaled and the corresponding displacement is changed, it can also be understood that the third point cloud information matches the first point cloud information. It can be understood that the point cloud information according to the sticker flattening algorithm sticker will also change according to the change of the first point cloud information, and the terminal will display the sticker according to the point cloud information obtained after the second point cloud information changes. Achieve the combination of stickers and portraits.

Optionally, as shown in FIG. 3(f), the terminal simultaneously captures two portraits, a portrait a and a portrait b, and the terminal generates a point cloud information and b point cloud information corresponding to the two portraits respectively. At this time, the terminal Will determine whether the point cloud information matches the first point cloud information and whether the point b cloud information matches the first point cloud information. The specific judgment method is as shown in Figure 3(c), 3(d) or 3(e). The judgment manners described in the corresponding embodiments are similar, and are not described herein again. Therefore, when the a point cloud information and the b point cloud information are matched with the first point cloud information, the terminal also has the corresponding positions of the portrait a and the portrait b. Displaying the sticker, it can be understood that if only the point cloud information matches the first point cloud information, then the terminal only displays the sticker in the corresponding position of the portrait a. If only the b point cloud information matches the first point cloud information, Then, the terminal displays the sticker only in the corresponding position of the portrait b. In addition, the number of the images captured by the terminal is not limited to the two examples. The actual application scenario is preferred, and is not limited herein.

The above describes the display manner of the definition sticker in the terminal. The data processing method of the present application is described in detail below in conjunction with the specific shooting process of the terminal in the actual application.

Referring to FIG. 4, another embodiment of the data processing method in the embodiment of the present application includes:

401. Start the camera app.

In this embodiment, the user operates the terminal to open the camera APP, and correspondingly, the camera of the terminal is turned on, and the camera sends the acquired image data to the camera APP.

402. Select a target SDK from the SDK set.

In this embodiment, the camera APP can integrate multiple SDKs including a sticker shooting algorithm, and the SDK integrated by the camera APP has a unified SDK interface. It can be understood that developers who need to provide these SDKs develop corresponding rules according to uniform rules. Shoot the algorithm and package the SDK.

It should be noted that the unified SDK interface may include a process of downloading and registering a sticker, initializing a shooting interface, performing a sticker preview, processing shooting data, and ending a process. It can be understood that the SDK interface described herein is only an example. The specific SDK interface in the actual application may be changed, and is not limited herein.

It should be noted that the SDK integrated by all camera APPs is a sticker material defined according to the first point cloud information of the standard portrait, that is to say, the sticker material provided by the third party needs to be defined according to the first point cloud information, and correspondingly, The developer is required to develop a sticker flattening algorithm based on the definition of the sticker. The specific manner is similar to that described in the embodiment shown in FIG. 2, and details are not described herein again.

The terminal can select the target SDK from the SDK set integrated by the camera APP by means of intelligent selection. Specifically, each SDK has a corresponding SDK parameter, and the parameter includes a parameter of the user skin level and a parameter of the preview algorithm processing rate. And the parameters of the sticker followability rate, and then the three parameters can be brought into the following formula to calculate the calculation result, and the calculation result with the largest value is selected from the set of calculation results, and the SDK corresponding to the calculation result is determined as the target. SDK.

The formula is R=α*B+γ*P+β*F, where R is the settlement result, B is the parameter of the user skin level, P is the parameter of the preview algorithm processing rate, and F is the The parameters of the sticker followability rate, α, β, and γ are preset weight coefficients. It should be noted that the preset weight coefficient can be different in different shooting modes. For example, in the camera mode, the requirement for skin color is higher, then the value of α can be increased accordingly, and the sticker followability rate in the recording mode. If the requirement is high, the value of β can be increased accordingly. The values of α, β, and γ are not limited here. The terminal can be built and the corresponding algorithm can be set and adjusted according to actual needs.

It should be noted that, in this embodiment, the camera APP can integrate multiple shooting algorithms in addition to multiple SDKs, that is, a shooting algorithm developed and disclosed by a developer of the shooting algorithm. In this scenario, the camera The APP no longer needs to call the SDK and can directly select a certain shooting algorithm to achieve various shooting functions such as portrait beauty, portrait detection, filter shooting and sticker flattening. The selection of the shooting algorithm can be selected by the terminal intelligently, or The user chooses it according to his or her preference, which is not limited here.

403. Determine whether the sticker selected by the user has been downloaded. If yes, execute step 406. If no, perform step 404.

In this embodiment, the terminal needs to determine whether the sticker selected by the user has been downloaded and registered to the sticker management module in the camera APP.

404. Download the sticker from the server.

In this embodiment, if the sticker selected by the user has not been downloaded, the camera APP can download the sticker selected by the user by accessing the server. It can be understood that the server may be a third-party server corresponding to the third-party SDK shown in FIG. 1 . For example, if the camera APP currently uses an SDK developed by Company A, then the camera app can download stickers from A company's private server.

It should be noted that, in this embodiment, the server may also be a public server. As shown in FIG. 5, in addition to downloading the sticker from the third-party server, the camera APP may also download other third-party designed stickers from the public server. The camera APP integrated SDK has a unified interface, and the SDK is a sticker material defined according to the first point cloud information of the standard portrait, so that the third-party sticker material on the public server is also the first point cloud according to the standard portrait. The information is defined, then any kind of SDK used by the current camera APP can present the sticker on the public server in the interface of the terminal.

For the specific scenario in the actual application, refer to FIG. 6. For example, the camera APP currently uses the SDK developed by the company A, and the A sticker, the B sticker, and the C sticker in the sticker material list are all designed and announced by the company A, and the current SDK can be Download new stickers and update the sticker material list from company A's private server according to user needs. In addition, users can also To download the sticker material published by the company, such as company A, company B or company C, from the public server by clicking the "+" button on the interface, and add it to the sticker list of the camera app for use.

405. Register the sticker to the sticker management module.

In this embodiment, the camera APP registers the sticker downloaded from the server to the local sticker management module for saving. When the user opens the camera APP for the next time to shoot, there is no need to repeatedly download the sticker material that has been used before.

406. Determine whether a portrait is detected. If yes, execute step 407. If no, repeat step 406.

In this embodiment, if the sticker selected by the user has been downloaded, the camera app will continue to perform the portrait detection until it is determined that the portrait has been detected, wherein the description of the portrait detection with respect to the camera APP and the steps in the embodiment shown in FIG. The descriptions of 203 and 204 are similar, and details are not described herein again.

407. Display a sticker selected by the user.

In this embodiment, when the camera APP determines that the portrait is detected, the second point cloud information of the sticker may be displayed on the interface of the terminal, wherein the description of the sticker according to the second point cloud information is shown in FIG. 2 The description of step 204 in the embodiment is similar, and details are not described herein again.

408. Perform a combination of portraits and stickers.

In this embodiment, the camera APP can adjust the display position or display size of the sticker according to the integrated SDK or sticker flattening algorithm, and finally complete the flattening of the sticker and the portrait.

Secondly, in the embodiment of the present application, the camera APP can integrate multiple SDKs or shooting algorithms, and all of the SDKs have a unified SDK interface, and are all defined according to the first point cloud information of the standard portrait, so as long as the public server The third-party sticker material is also defined according to the first point cloud information of the standard portrait. Then, any SDK used by the current camera APP can present the sticker on the public server on the interface of the terminal, and the user is no longer limited to only Users can download stickers from a third-party private server, and users can download any published stickers from the public server, which makes the user's choice of stickers more rich and enhances the user experience.

The following describes the terminal in the embodiment of the present application:

Referring to FIG. 7, an embodiment of a terminal in this embodiment of the present application includes:

a first acquiring unit 701, configured to acquire first point cloud information of a standard portrait, where the first point cloud information is used to indicate coordinates of the standard portrait on a display screen of the terminal;

a second acquiring unit 702, configured to acquire second point cloud information of the sticker, the second point cloud information is at least a part of the first point cloud information, and the second point cloud information is used to indicate the Sticker relative to the standard portrait Position, the second point cloud information is artificially formulated, and the sticker is a pattern that can be displayed in the display screen;

The generating unit 703 is configured to generate third point cloud information when the terminal detects the portrait in the shooting preview state, where the third point cloud information is used to indicate that the terminal is in the shooting preview state The coordinates on the screen;

The display unit 704 is configured to display the sticker according to the second point cloud information when the third point cloud information matches the first point cloud information and the user selects the sticker.

In the technical solution provided by the embodiment of the present application, the first acquiring unit 701 may obtain the first point cloud information of the standard portrait, the second acquiring unit 702 may obtain the second point cloud information, and the second point cloud information is used to indicate that the sticker is opposite to the sticker. When the standard portrait is located, when the portrait is detected, the generating unit 703 may generate the third point cloud information of the portrait, and further if the third point cloud information matches the first point cloud information and the user selects the sticker, the display unit 704 The sticker will be displayed according to the second point cloud information. Since the second point cloud information of the sticker is artificially determined according to the content of the sticker, for example, the sticker content is a pair of glasses, then the sticker is near the eye of the standard portrait, and the sticker content If it is a hat, then the sticker is near the top of the standard portrait, so when the third point cloud information matches the first point cloud information, the second point cloud information of the sticker is the initial display position of the sticker and the sticker. In the actual scene, the error between the actual display position and the portrait is small, which saves the time for the subsequent terminal to adjust the position of the sticker. The user experience.

For the sake of understanding, the terminal in the embodiment of the present application is described in detail below. Referring to FIG. 8 , another embodiment of the terminal in this embodiment of the present application includes:

a first acquiring unit 801, configured to acquire first point cloud information of a standard portrait, where the first point cloud information is used to indicate coordinates of the standard portrait on a display screen of the terminal;

a second acquiring unit 802, configured to acquire second point cloud information of the sticker, where the second point cloud information is at least a part of the first point cloud information, where the second point cloud information is used to indicate the The second point cloud information is artificially defined by the position of the sticker relative to the standard portrait, and the sticker is a pattern that can be displayed in the display screen;

The generating unit 803 is configured to generate third point cloud information when the terminal detects the portrait in the shooting preview state, where the third point cloud information is used to indicate that the terminal is in the shooting preview state The coordinates on the screen;

The display unit 804 is configured to display the sticker according to the second point cloud information when the third point cloud information matches the first point cloud information and the user selects the sticker.

The first selection unit 805 is configured to select a target SDK from the SDK set, and all the SDKs in the SDK set have a unified SDK interface, and the target SDK is used to implement the stitching of the portrait and the sticker.

The second selecting unit 806 is configured to select a target shooting algorithm from the set of shooting algorithms, and the target shooting algorithm is used to implement the stitching of the portrait and the sticker.

The first selection unit 805 in the embodiment of the present application further includes:

The obtaining module 8051 is configured to obtain an SDK parameter set corresponding to the SDK set, where the SDK parameter set includes a parameter set of a user skin level, a parameter set of a preview algorithm processing rate, or a parameter set of a sticker followability rate;

a calculation module 8052, configured to substitute the SDK parameter set into a formula to obtain a calculation result set;

a first determining module 8053, configured to determine, from the set of calculation results, a target calculation result, where the target calculation result is a calculation result having the largest value in the calculation result set;

The second determining module 8054 is configured to determine the target SDK corresponding to the target calculation result.

The terminal in the embodiment of the present application is described above from the perspective of a modular functional entity, and is processed from hardware below. The terminal in the embodiment of the present application is described. Referring to FIG. 9, another embodiment of the terminal in this embodiment of the present application includes:

The embodiment of the present application further provides another terminal. As shown in FIG. 9 , for the convenience of description, only the part related to the embodiment of the present application is shown. If the specific technical details are not disclosed, please refer to the method part of the embodiment of the present application. . The terminal can be any terminal including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, and the terminal is a mobile phone as an example:

FIG. 9 is a block diagram showing a partial structure of a mobile phone related to a terminal provided by an embodiment of the present application. Referring to FIG. 9, the mobile phone includes: a radio frequency (RF) circuit 910, a memory 920, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a wireless fidelity (WiFi) module 970, and a processor 980. And power supply 990 and other components. It will be understood by those skilled in the art that the structure of the handset shown in FIG. 9 does not constitute a limitation to the handset, and may include more or less components than those illustrated, or some components may be combined, or different components may be arranged.

The following describes the components of the mobile phone in detail with reference to FIG. 9:

The RF circuit 910 can be used for receiving and transmitting signals during the transmission or reception of information or during a call. Specifically, after receiving the downlink information of the base station, it is processed by the processor 980. In addition, the uplink data is designed to be sent to the base station. Generally, RF circuit 910 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, RF circuitry 910 can also communicate with the network and other devices via wireless communication. The above wireless communication may use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division). Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), E-mail, Short Messaging Service (SMS), and the like.

The memory 920 can be used to store software programs and modules, and the processor 980 executes various functional applications and data processing of the mobile phone by running software programs and modules stored in the memory 920. The memory 920 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored according to Data created by the use of the mobile phone (such as audio data, phone book, etc.). Moreover, memory 920 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 930 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function controls of the handset. Specifically, the input unit 930 may include a touch panel 931 and other input devices 932. The touch panel 931, also referred to as a touch screen, can collect touch operations on or near the user (such as a user using a finger, a stylus, or the like on the touch panel 931 or near the touch panel 931. Operation), and drive the corresponding connecting device according to a preset program. Optionally, the touch panel 931 can include two parts: a touch detection device and a touch controller. Wherein, the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information. The processor 980 is provided and can receive commands from the processor 980 and execute them. In addition, The touch panel 931 can be implemented in various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 931, the input unit 930 may also include other input devices 932. Specifically, other input devices 932 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like.

The display unit 940 can be used to display information input by the user or information provided to the user as well as various menus of the mobile phone. The display unit 940 can include a display panel 941. Alternatively, the display panel 941 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. Further, the touch panel 931 can cover the display panel 941. When the touch panel 931 detects a touch operation on or near it, the touch panel 931 transmits to the processor 980 to determine the type of the touch event, and then the processor 980 according to the touch event. The type provides a corresponding visual output on display panel 941. Although the touch panel 931 and the display panel 941 are used as two independent components to implement the input and input functions of the mobile phone in FIG. 9, in some embodiments, the touch panel 931 and the display panel 941 may be integrated. Realize the input and output functions of the phone.

The handset may also include at least one type of sensor 950, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 941 according to the brightness of the ambient light, and the proximity sensor may close the display panel 941 and/or when the mobile phone moves to the ear. Or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity. It can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for the mobile phone can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, no longer Narration.

An audio circuit 960, a speaker 961, and a microphone 962 can provide an audio interface between the user and the handset. The audio circuit 960 can transmit the converted electrical data of the received audio data to the speaker 961, and convert it into a sound signal output by the speaker 961. On the other hand, the microphone 962 converts the collected sound signal into an electrical signal, and the audio circuit 960 After receiving, it is converted into audio data, and then processed by the audio data output processor 980, sent to the other mobile phone via the RF circuit 910, or outputted to the memory 920 for further processing.

WiFi is a short-range wireless transmission technology, and the mobile phone can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 970, which provides users with wireless broadband Internet access. Although FIG. 9 shows the WiFi module 970, it can be understood that it does not belong to the essential configuration of the mobile phone, and can be omitted as needed within the scope of not changing the essence of the application.

The processor 980 is the control center of the handset, which connects various portions of the entire handset using various interfaces and lines, by executing or executing software programs and/or modules stored in the memory 920, and invoking data stored in the memory 920, executing The phone's various functions and processing data, so that the overall monitoring of the phone. Optionally, the processor 980 may include one or more processing units; preferably, the processor 980 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application, and the like. The modem processor primarily handles wireless communications. It will be appreciated that the above described modem processor may also not be integrated into the processor 980.

The handset also includes a power source 990 (such as a battery) that powers the various components. Preferably, the power source can be logically coupled to the processor 980 via a power management system to manage charging, discharging, and power management through the power management system. And other functions.

Although not shown, the mobile phone may further include a camera, a Bluetooth module, and the like, and details are not described herein again.

In the embodiment of the present application, the processor 980 included in the terminal further has the following functions:

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application, in essence or the contribution to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program code. .

The above embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still The technical solutions described in the embodiments are modified, or the equivalents of the technical features are replaced by the equivalents. The modifications and substitutions of the embodiments do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A data processing method, comprising:

The terminal acquires first point cloud information of the standard portrait, where the first point cloud information is used to indicate coordinates of the standard portrait on a display screen of the terminal;

The terminal acquires second point cloud information of the sticker, the second point cloud information is at least a part of the first point cloud information, and the second point cloud information is used to indicate the sticker relative to the a position of the standard portrait, the second point cloud information is artificially formulated, and the sticker is a pattern that can be displayed in the display screen;

When the terminal detects a portrait in a shooting preview state, the terminal generates third point cloud information, where the third point cloud information is used to indicate that the terminal is in the display screen in a shooting preview state. Coordinate on

When the third point cloud information matches the first point cloud information and the user selects the sticker, the terminal displays the sticker according to the second point cloud information.
The method according to claim 1, wherein before the generating, by the terminal, the third point cloud information, the method further comprises:

The terminal selects a target SDK from the software development kit SDK set, and all the SDKs in the SDK set have a unified SDK interface, and the target SDK is used to implement the flattening of the portrait and the sticker.
The method according to claim 2, wherein the selecting, by the terminal, the target SDK from the set of SDKs comprises:

The terminal acquires an SDK parameter set in the SDK set, where the SDK parameter set includes a parameter set of a user skin level, a parameter set of a preview algorithm processing rate, and a parameter set of a sticker followability rate;

The terminal obtains a calculation result set according to the SDK parameter set;

Determining, by the terminal, a target calculation result from the set of calculation results, where the target calculation result is a calculation result having the largest value in the calculation result set;

The terminal determines the target SDK corresponding to the target calculation result.
The method according to claim 3, wherein the terminal obtains a set of calculation results according to the set of SDK parameters, including:

The terminal calculates the calculation result set by bringing the SDK parameter set into the following formula, and the formula is:

R=α*B+γ*P+β*F;

The R is a settlement result;

The α, β, and γ are preset weight coefficients;

The B is a parameter of the user's skin level;

The P is a parameter of a processing rate of the preview algorithm;

The F is a parameter of the sticker followability rate.
The method according to any one of claims 2 to 4, wherein the unified SDK interface comprises:

Download and register stickers, capture interface initialization, perform sticker previews, process shooting data, and end the process.
The method according to claim 1, wherein before the generating, by the terminal, the third point cloud information, the method further comprises:

The terminal selects a target shooting algorithm from a set of shooting algorithms, and the target shooting algorithm is used to implement the portrait The combination with the sticker.
The method according to claim 6, wherein the target shooting algorithm comprises:

Portrait beauty algorithm, portrait detection algorithm, filter shooting algorithm and/or sticker flattening algorithm.
A terminal, comprising:

a first acquiring unit, configured to acquire first point cloud information of a standard portrait, where the first point cloud information is used to indicate coordinates of the standard portrait on a display screen of the terminal;

a second acquiring unit, configured to acquire second point cloud information of the sticker, where the second point cloud information is at least a part of the first point cloud information, and the second point cloud information is used to indicate the sticker The second point cloud information is artificially formulated relative to the location of the standard portrait, and the sticker is a pattern that can be displayed in the display screen;

a generating unit, configured to generate third point cloud information when the terminal detects a portrait in a shooting preview state, where the third point cloud information is used to indicate that the terminal is in the shooting preview state The coordinates on the display;

a display unit, configured to display the sticker according to the second point cloud information when the third point cloud information matches the first point cloud information and the user selects the sticker.
The terminal according to claim 8, wherein the terminal further comprises:

And a first selection unit, configured to select a target SDK from the SDK set, all SDKs in the SDK set have a unified SDK interface, and the target SDK is used to implement the stitching of the portrait and the sticker.
The terminal according to claim 9, wherein the first selection unit comprises:

An acquiring module, configured to acquire an SDK parameter set corresponding to the SDK set, where the SDK parameter set includes a parameter set of a user skin level, a parameter set of a preview algorithm processing rate, and a parameter set of a sticker followability rate;

a calculation module, configured to substitute the SDK parameter set into a formula to calculate a calculation result set;

a first determining module, configured to determine a target calculation result from the set of calculation results, where the target calculation result is a calculation result with the largest value in the calculation result set;

And a second determining module, configured to determine the target SDK corresponding to the target calculation result.
The terminal according to claim 8, wherein the terminal further comprises:

And a second selecting unit, configured to select a target shooting algorithm from the set of shooting algorithms, where the target shooting algorithm is used to implement the stitching of the portrait and the sticker.
A terminal, comprising:

a processor, a memory, a bus, and an input and output interface;

Program code is stored in the memory;

The processor performs the following operations when calling the program code in the memory:

Obtaining first point cloud information of the standard portrait, the first point cloud information is used to indicate coordinates of the standard portrait on a display screen of the terminal;

Obtaining a second point cloud information of the sticker, the second point cloud information is at least a part of the first point cloud information, and the second point cloud information is used to indicate that the sticker is relative to the standard portrait Position, the second point cloud information is artificially formulated, and the sticker is a pattern that can be displayed in the display screen;

When the terminal detects a portrait in the shooting preview state, generating third point cloud information, the third point cloud information is used to indicate coordinates of the portrait on the display screen in the shooting preview state of the terminal ;

When the third point cloud information matches the first point cloud information and the user selects the sticker, the sticker is displayed according to the second point cloud information.
A computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 7.
A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 7.