WO2022142388A1 - Special effect display method and electronic device - Google Patents

Abstract

The present disclosure relates to the field of information display, and relates to a special effect display method and an electronic device. The method comprises: obtaining spatial position information of an image acquisition device at the current moment; determining a display parameter of a target special effect object on the basis of the spatial position information, the target special effect object being used for processing a picture acquired by the image acquisition device; and rendering the picture on the basis of the display parameter, and displaying a special effect picture obtained by the rendering, the special effect picture comprising the target special effect object.

Description

Special effect display method and electronic device

The present disclosure is based on the Chinese patent application with the application number of 202011599632.6 and the filing date of December 29, 2020, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated herein by reference.

technical field

The present disclosure relates to the field of information display, and in particular, to a special effect display method and an electronic device.

Background technique

At this stage, in image capture devices with shooting functions such as mobile phones, native cameras or third-party shooting applications usually provide users with special effects such as magic expressions, so that users can invoke these special effects during the shooting process to achieve a variety of shooting effects.

However, in the related art, the activation and display of special effects are usually realized by means of timing triggering or expression triggering. For example, under the timing trigger mechanism, the next special effect object is automatically displayed after the current special effect object is displayed for 3 seconds.

SUMMARY OF THE INVENTION

According to an aspect of the embodiments of the present disclosure, a special effect display method is provided, including:

Obtain the spatial location information of the image acquisition device at the current moment;

determining the display parameters of the target special effect object based on the spatial position information, and the target special effect object is used to process the picture captured by the image capture device;

The picture is rendered based on the display parameters, and the special effect picture obtained by rendering is displayed, and the special effect picture includes the target special effect object.

According to another aspect of the embodiments of the present disclosure, a special effect display device is provided, including:

an information acquisition unit, configured to acquire the spatial position information of the image acquisition device at the current moment;

a parameter determination unit, configured to determine display parameters of a target special effect object based on the spatial position information, where the target special effect object is used to process the picture captured by the image capture device;

A picture rendering unit, configured to render the picture based on the display parameters, and display a special effect picture obtained by rendering, and the special effect picture includes the target special effect object.

According to another aspect of the embodiments of the present disclosure, an electronic device is provided, including:

processor;

a memory for storing the processor-executable instructions;

Wherein, the processor is configured to execute the instructions to achieve the following steps:

Obtain the spatial location information of the image acquisition device at the current moment;

determining the display parameters of the target special effect object based on the spatial position information, and the target special effect object is used to process the picture captured by the image capture device;

The picture is rendered based on the display parameters, and the special effect picture obtained by rendering is displayed, and the special effect picture includes the target special effect object.

According to another aspect of the embodiments of the present disclosure, a storage medium is provided. When an instruction in the storage medium is executed by a processor of an electronic device, the electronic device can perform the following steps:

Obtain the spatial location information of the image acquisition device at the current moment;

determining the display parameters of the target special effect object based on the spatial position information, and the target special effect object is used to process the picture captured by the image capture device;

The picture is rendered based on the display parameters, and the special effect picture obtained by rendering is displayed, and the special effect picture includes the target special effect object.

According to another aspect of the embodiments of the present disclosure, a computer program product is provided, comprising a computer program and/or instructions, the computer program and/or instructions implement the following steps when executed by a processor:

Obtain the spatial location information of the image acquisition device at the current moment;

determining the display parameters of the target special effect object based on the spatial position information, and the target special effect object is used to process the picture captured by the image capture device;

The picture is rendered based on the display parameters, and the special effect picture obtained by rendering is displayed, and the special effect picture includes the target special effect object.

According to the embodiment of the present disclosure, the user controls the display effect of the special effect by manipulating the spatial posture of the image capturing device. In some embodiments, the display parameters of the target special effect object are determined by acquiring the spatial position information of the image acquisition device at the current moment, so that the captured picture is rendered based on the display parameters to obtain the special effect picture, so that the special effect picture can be displayed when displayed. The above-mentioned special effect object is displayed, thereby presenting a special effect display effect corresponding to the spatial position information of the image acquisition device. It can be seen that this solution allows the user to change the spatial position information of the image capture device by manipulating the image capture device, and then adjust and control the special effects such as magic expressions displayed by the image capture device; and this solution does not need to use the user's face key Points and other biological information can be realized, which not only ensures the control accuracy of the display effect, but also ensures the user's information security.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure and do not unduly limit the present disclosure.

1 is a schematic diagram of a special effect display scene provided by an exemplary embodiment;

2 is a flowchart of a method for displaying special effects provided by an exemplary embodiment;

3 is a flowchart of another special effect display method provided by an exemplary embodiment;

4 is a schematic diagram of a magic expression display effect shown in an exemplary embodiment;

5 is a schematic diagram of another magic expression display effect shown in an exemplary embodiment;

FIG. 6 is a schematic diagram of another magic expression display effect shown in an exemplary embodiment;

7 is a schematic block diagram of a special effect display device according to an exemplary embodiment;

FIG. 8 is a structural diagram of an electronic device according to an exemplary embodiment.

Detailed ways

In order to make those skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first", "second" and the like in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the disclosure described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

As used in this disclosure, the terms "at least one", "plurality", "each", at least one includes one, two or more, multiple includes two or more, and each refers to the corresponding each of the multiple. The user information involved in this disclosure is the information authorized by the user or fully authorized by all parties.

At this stage, in image capture devices with shooting functions such as mobile phones, native cameras or third-party shooting applications usually provide users with special effects such as magic expressions, so that users can invoke these special effects during the shooting process to achieve a variety of shooting effects.

However, in the related art, the activation and display of special effects are usually realized by means of timing triggering or expression triggering. For example, under the timing trigger mechanism, the next special effect object is automatically displayed after the current special effect object is displayed for 3 seconds. At this time, it is often difficult for the user to control the special effect display state; while under the expression trigger mechanism, the user can control the special effect display through facial expressions. However, this method has a strong dependence on the user's face key point data and other biological information, so there may be problems such as inaccurate control or information security.

As shown in Figure 1, when the user uses an image acquisition device (such as a mobile phone) with an image acquisition function (such as a mobile phone) to shoot, the special effects provided by the mobile phone are turned on, so that the real-time data captured by the camera is displayed on the mobile phone display. At the same time as the picture to be shot is displayed, the corresponding special effects objects are superimposed and displayed, such as the magic expression in the shape of a rabbit ear in Figure 1 (hereinafter referred to as the rabbit ear). At this time, the user can control the magic expression through a timing trigger mechanism or by making a specific facial expression.

To this end, the present disclosure proposes a new way of playing the special effect function, that is, through the spatial position information of the image capture device, the display effect of the target special effect object displayed by the image capture device is controlled, so that the user using the image capture device can control the posture of the image capture device by controlling the image capture device. , to realize self-adjustment and control of the display effect of the target special effect object, which is displayed in the picture collected by the image acquisition device.

FIG. 2 is a flowchart of a special effect display method according to an exemplary embodiment. As shown in Figure 2, the method is performed by an image acquisition device, and includes the following steps:

Step 202: Obtain the spatial position information of the image acquisition device at the current moment.

The image acquisition device periodically acquires spatial position information in the process of acquiring images. In this embodiment, the image acquisition device is equipped with image acquisition components such as a camera module, and is also equipped with image display components such as a display screen. For example, the image acquisition device is a smartphone, and the image acquisition component assembled on the smartphone is the front and/or rear camera, and the image display component of the smartphone is a liquid crystal display panel or a light emitting diode (Light Emitting Diode, LED) display panel. Of course, in addition to mobile phones, image acquisition devices are tablet devices, notebook computers, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smart watches, etc.) The description of one or more embodiments is not limiting in this regard.

In this embodiment, the image acquisition device acquires its own spatial position information during the process of acquiring images, such as in the image preview stage before the image is captured, the image preview stage before the video is captured, or in the process of capturing the video. The position information is used to process the captured picture, so as to display the target special effect object corresponding to the spatial position information in the preview picture or the shooting picture.

In this embodiment, the image acquisition device perceives its own spatial posture through a sensor, so that in the process of acquiring a picture, the sensor is used to obtain its own spatial position information at the current moment, wherein the sensor is assembled in the image acquisition device, as shown in this The sensor is a gyroscope assembled by the image acquisition device itself; or the sensor is connected to The image acquisition device, for example, the sensor is a laser rangefinder connected to the image acquisition device, etc., will not be described in detail.

In some embodiments, there are various connection relationships between the above sensors and the image acquisition device, and accordingly, the image acquisition device acquires the spatial position information acquired by the sensor in different ways. In some embodiments, the above-mentioned sensor is an integrated sensor assembled in an image capture device. In this case, the “image capture device” should be understood as not including the software and hardware parts of the sensor. Correspondingly, a control component such as a central processing unit (CPU) in the image acquisition device sends an acquisition request to the sensor, and the acquisition request is used to acquire the spatial position information of the image acquisition device collected by the sensor. The collected spatial position information of the image device is sent to the image capture device (or the CPU of the image capture device). Or the sensor periodically provides the spatial position information collected by the sensor to the image collection device. At this time, the sensor assembled by the image capture device itself is used to collect the spatial position information of the image capture device, so that the user can adjust the special effects displayed by the image capture device by manipulating the spatial posture of the image capture device. As shown in Figure 1, the sensor is assembled in a fixed position inside the mobile phone.

In some embodiments, the sensor involved in this solution includes at least one of the following: an acceleration sensor (G-Sensor, which outputs acceleration values in three directions) for sensing acceleration in three directions of x, y, and z. , Gyro sensor (Gyro-sensor, referred to as gyroscope, outputs the acceleration values in the three directions of x, y, and z), the direction sensor for sensing the angle (O-Sensor, outputs the three directions of x, y, and z) angle), a linear acceleration sensor (LA-Sensor, which outputs the acceleration value after removing the influence of gravity), a linear acceleration sensor (LA-Sensor, which outputs the acceleration value after removing the influence of gravity), a rotation vector sensor (RV-Sensor, which is used to perceive the rotation angle, and the output is calculated by mixing the coordinate axis and the angle) The quaternion and other data), the gravity sensor (GV-Sensor, output gravitational acceleration value) for sensing the gravity at the location, the sensor (M-sensor, output x, y, z) for sensing the environmental magnetic field at the location The three-axis ambient magnetic field strength). Among them, the output values of LA-Sensor, RV-Sensor and GV-Sensor are calculated by G-sensor, O-sensor and Gyro-sensor through preset algorithms. No restrictions are imposed.

In some embodiments, the spatial position information is used to indicate the spatial posture of the image acquisition device at the current moment, which includes at least one of acceleration, rotation angle, quaternion, etc. Of course, according to the requirements of the actual display effect, it also includes other information, the embodiments of the present disclosure do not limit this.

In some embodiments, the spatial position information collected by the image capture device through the sensor may include multiple types, and only some of them are required for the target special effect object to be displayed, so the image capture device determines the spatial position information to be acquired. Correspondingly, the step of acquiring the spatial position information of the image acquisition device at the current moment by the image acquisition device includes: the image acquisition device determines a target type, and the target type is the type of spatial position information required to display the target special effect object; based on the target type , to determine the spatial location information.

Wherein, the step of determining the target type by the image acquisition device includes: the image acquisition device determines the target type by querying a preset relationship table, and the preset relationship table is used to record the corresponding relationship between the special effect object and the target type, wherein any one of Several spatial position information of the target type corresponding to the special effect object is used to determine the display parameters of any special effect object.

The preset relationship table is recorded in the Magic Watch SDK (Software Development Kit, software development kit), so that the target type can be determined by calling the Magic Watch SDK. In this way, the image acquisition device can not only determine the target type accurately and efficiently by looking up the table, but also obtain the required spatial position information from the various (or at least one) spatial information of the image acquisition device at the current moment, In addition, it is not necessary to obtain all the spatial information of the image acquisition device, but only the part required to display the target special effect object, which helps to reduce the amount of data to be processed, speed up the display speed of the target special effect object, and avoid the special effect display being stuck, which is helpful for Improve the display effect.

In some embodiments, before the image capture device determines the target type by querying the preset relationship table, the image capture device needs to determine the target special effect object to be displayed. The image acquisition device determines the target special effect object to be displayed in various ways. In some embodiments, the image capture device identifies the display object in the captured picture, and determines the candidate special effect object that matches the display object as the target special effect object to be displayed, so that the target special effect object is the same as that contained in the captured picture. The corresponding display objects, enhance the sense of coordination of the screen, and can automatically display suitable special effects objects for users. For example, in the case of recognizing a female face, the image acquisition device determines the cat face special effect as the matching target special effect object; and in the case of detecting the night sky, determines the fireworks special effect as the matching target special effect object, etc.

In some embodiments, the image capturing device provides the user with a variety of candidate special effect objects, so that the user can select a suitable candidate special effect object for display. Correspondingly, in response to the special effect selection instruction for the candidate special effect object, the image acquisition device determines the selected candidate special effect object as the target special effect object to be displayed. In this way, the user can select the target special effect object according to his own wishes, so as to fully respect the user's choice.

The target special effect object is a static or dynamic magic expression displayed on the top of the screen. As shown in Figure 1, the magic expression is the rabbit ear displayed on the head of the portrait. At this time, the solution adjusts the display position, display color, rabbit ear size, rabbit ear size, rabbit ear, etc. At least one display attribute such as ear shape, rabbit ear motion, etc. In some embodiments, the image capture device provides the user with a variety of special effects objects for the user to choose, for example, in addition to the rabbit ears, it also provides other forms of cat face, dog head, red green blue (RGB) color separation, etc. The special effect object also provides virtual special effects that can be controlled by the user through interactive actions, such as a dynamic three-dimensional (Three Dimensions, 3D) display model, which is not limited in the embodiments of the present disclosure.

In some embodiments, the target special effect object and the image captured by the image capture device are displayed in the same display interface. For example, if the image captured by the camera is received on the screen of the image capture device, the image captured by the camera will be refreshed according to a preset refresh frequency (usually the image captured by the camera). Refresh frequency) to display, and when the target special effect object is displayed in the current frame image, the image acquisition device collects various spatial information through the sensor, and then determines the display parameters of the next frame image according to the spatial position information (such as the target The pixel value of each pixel point corresponding to the special effect object), so that the target special effect object (for example, a magic expression) corresponding to the spatial position of the mobile phone is displayed in the next frame of image.

In some embodiments, the image acquisition device acquires multiple (at least one) types of spatial position information at the current moment collected by the sensor, and then, based on the target type, determines from the multiple (at least one) types of spatial position information where the target special effect object is displayed. The required spatial position information, the target type is the type of spatial position information required to display the target special effect object, and the process is:

The sensor provides the collected various (or at least one) spatial position information of the image acquisition device at the current moment to the image acquisition device in real time or periodically, so that the image acquisition device receives various spatial position information provided by the sensor. Then, based on the special effect object to be displayed at the current moment, query the preset relationship table to determine the target type, and based on the target type, determine the spatial position from the multiple (or at least one) spatial position information of the image acquisition device at the current moment The spatial position information is the spatial position information required to display the target special effect object, so that the image acquisition device only needs to determine the part of the information to be used from all the received spatial position information.

In some embodiments, the image acquisition device instructs the sensor to acquire the spatial location information required to display the target special effect object. In this case, the sensor does not acquire the spatial location information in real time, but the image acquisition device sends an acquisition request to the sensor after determining the target type, The obtaining request carries a target type, and the target type is the type of spatial position information required to display the target special effect object. The sensor receives the acquisition request (ie calls the sensor), collects the spatial location information of the target type based on the target type, sends the spatial location information to the image acquisition device, and the image acquisition device receives the spatial location information. This method calls the sensor to collect spatial position information only when the image acquisition device needs it, thereby helping to reduce the frequency of use of the sensor to reduce power consumption.

Step 204: Determine the display parameters of the target special effect object based on the spatial position information, where the target special effect object is used to process the picture captured by the image capture device.

In some embodiments, the above-mentioned spatial position information includes quaternions. At this time, the image acquisition device constructs a three-dimensional rotation matrix, converts the quaternions into Euler angles based on the three-dimensional rotation matrix, and then determines the target special effect based on the Euler angles. Display parameters of the object, to achieve quick determination of display parameters. For example, the image acquisition device determines the Euler angle as a display parameter of the target special effect object.

Or, the image acquisition device determines the display parameters of the target special effect object based on the result of the product of the Euler angle and the preset multiple. For example, the image acquisition device determines the product of the Euler angle and the preset multiple as the display parameter of the target special effect object. The preset multiple is a positive number not equal to 1, so as to prevent the Euler angle from being too large or too small, resulting in poor display of the final target special effect object. Wherein, when the Euler angle is too small, the preset multiple should be greater than 1; and when the Euler angle is too large, the preset multiple should be less than 1, so as to realize the optimal adjustment of the display effect . Wherein, when the Euler angle is less than the first preset angle, it is determined that the Euler angle is too small; and/or when the Euler angle is greater than the second preset angle, it is determined that the Euler angle is too large, and the first The specific values of the preset angle and the second preset angle are preset according to the actual situation such as the device type, the size of the display interface, the type of the target special effect object, etc. For example, the first preset angle is set to 0.5°, and the second preset angle is set to 80°. etc., the embodiments of the present disclosure do not limit this.

In some embodiments, the display parameters of the target special effect object are determined periodically, and in any period of determining the display parameters of the target special effect object, the Euler angle used is a fixed value; then in the current period, the After the quaternion is converted into Euler angles, the image acquisition device does not use the above Euler angles or their preset multiples as display parameters, but further determines by the following methods: obtaining at least one historical product result determined in adjacent historical periods, The adjacent historical period includes at least one continuous period before the current period, then determine the maximum value or average value of the product result in the current period and the above-mentioned at least one historical product result, and determine the maximum value or average value as the target special effect Object's display parameters.

Alternatively, the image acquisition device further determines the display parameters by obtaining at least one historical Euler angle determined in an adjacent historical period, where the adjacent historical period includes at least one period that is located before the current period and is continuous, and determines that within the current period The maximum value or average value of the Euler angle and the above at least one historical Euler angle, and the maximum value or average value is determined as the display parameter of the target special effect object. In the above manner, the image acquisition device determines the maximum value or average value of the current Euler angle and the historical Euler angle as the display parameter of the target special effect object, or determines the maximum value or average value of the current product result and the historical product result as the target Display parameters of the effect object. However, the above average value or maximum value is an absolute value. In the embodiment of the present disclosure, using the above average value as the display parameter of the target special effect object can ensure that the display effect of the target special effect object between adjacent image frames displayed in the display interface of the image acquisition device will not change abruptly, thereby avoiding sudden changes in the display effect of the target special effect object. The target special effect object undergoes a screen jump. Using the above maximum value as the display parameter of the target special effect object can ensure that the target special effect object is dynamically displayed as much as possible, and enhance the display interest of the target special effect object.

In some embodiments, the above-mentioned spatial position information includes acceleration. In this case, the image acquisition device first determines a special effect display area, and the special effect display area is used to display the target special effect object, and then determines the pixel characteristic value in the special effect display area according to the above acceleration, and sets the The pixel feature value is determined as the display parameter of the target special effect object. The pixel feature value is used to represent the feature of the corresponding pixel point; for example, the pixel feature value includes at least one of the following: color value, color gradient, transparency, and contrast. For example, the above-mentioned color gradient value is positively correlated with acceleration, that is, the greater the acceleration, the greater the color gradient value, and the smaller the acceleration, the smaller the color gradient value; The smaller the contrast, the smaller the contrast, and will not be repeated. In this way, the display parameters of the target special effect object can fully reflect the acceleration of the image acquisition device at the current moment, so that the display effect of the target special effect object corresponds to the acceleration, and the display of the target special effect object can be realized by controlling the spatial position of the image acquisition device. The purpose of adjusting the effect.

In some embodiments, the above-mentioned spatial position information includes a rotation angle, and the target special effect object has a preset standard display angle. In this case, the image acquisition device determines the deflection angle of the target special effect object based on the rotation angle. The display angle is rotated and transformed, and the angle after the rotation transformation is determined as the display parameter of the target special effect object. For example, when the standard display angle of the rabbit ear is 0°, the rabbit ear will show a vertical display state of the center line of the two ears under normal state, but when the user changes the spatial position of the mobile phone, the mobile phone rotates (the rotation angle is generated), The component of the rotation angle along the central axis of the mobile phone screen also changes, that is, a deflection angle is generated. At this time, the mobile phone is equivalent to rotating the deflection angle (such as 30°) along the central axis of the mobile phone screen. Also rotated 30° relative to the phone. At this time, the image acquisition device rotates and transforms the standard display angle according to the rotation angle, that is, the angle is added by 30 degrees, so that the rotated angle of the rabbit ears is 30 degrees, so that the rabbit ears are still displayed on the face of the human, and the The relative positional relationship between faces remains unchanged. It can be seen that this method can ensure that the display effect of the target special effect object matches the relative positional relationship between the picture content in the picture captured by the image acquisition device, and the display of the target special effect object will not be chaotic due to the rotation of the mobile phone.

In some embodiments, the display parameters of the target special effect object include blur intensity and/or color saturation, and also include at least one of feathering, contrast, brightness, etc., which are not limited in the embodiments of the present disclosure. In addition, the processing logic of the above embodiments is recorded in a pre-generated special effect SDK, so that the special effect SDK can be called to determine the display parameters of the target special effect object based on the spatial position information. For example, the special effect SDK is the aforementioned magic watch SDK, that is, the magic watch SDK not only records the corresponding relationship between the special effect object and the target type, but also records the processing logic and specific algorithm for solving the display parameters of the target special effect object through the spatial position information. Therefore, after acquiring the spatial position information, the image capture device provides it to the preset special effects SDK, so that the special effects SDK determines the display parameters of the target special effect object according to the processing logic and specific algorithm recorded therein, and then the image capture device Get this display parameter to display the target special effect object.

Among them, the above-mentioned process of determining the display parameters of the target special effect object through the Magic Watch SDK is the CPU of the image acquisition device, and the spatial location information is determined according to the above-mentioned corresponding relationship recorded by the Magic Watch SDK, and then according to the processing logic and the processing logic recorded in the Magic Watch SDK. The specific algorithm is the process of solving the display parameters of the target special effect object. By pre-setting the special effects SDK, the process of obtaining spatial position information and the method of determining display parameters can be standardized, which is convenient for determining display parameters quickly and efficiently, thereby ensuring the rapid rendering and display of special effects objects, and reducing the waiting time of users when special effects are displayed. user experience.

Step 206: Render the picture based on the display parameters, and display the special effect picture obtained by rendering, where the special effect picture includes the target special effect object.

In this embodiment, after the above-mentioned display parameters are determined, the image acquisition device provides them to an image processor (Graphic Processing Unit, GPU) corresponding to the display component, so that the GPU performs corresponding rendering processing based on the display parameters, and obtains The special effect picture of the target special effect object, and then the special effect picture is displayed, and the target special effect object in the special effect picture presents a display effect corresponding to the spatial position information.

According to the embodiment of the present disclosure, the user controls the display effect of the special effect by manipulating the spatial posture of the image capturing device. In some embodiments, the display parameters of the target special effect object are determined by acquiring the spatial position information of the image acquisition device at the current moment, so that the captured picture is rendered based on the display parameters to obtain the special effect picture, so that the special effect picture can be displayed when displayed. The above-mentioned special effect object is displayed, thereby presenting a special effect display effect corresponding to the spatial position information of the image acquisition device. It can be seen that this solution allows the user to change the spatial position information of the image capture device by manipulating the image capture device, and then adjust and control the special effects such as magic expressions displayed by the image capture device; and this solution does not need to use the user's face key Points and other biological information can be realized, which not only ensures the control accuracy of the display effect, but also ensures the user's information security.

The following is a flow chart of another special effect display method shown in FIG. 3 . The specific process of controlling the display effect of the magic expression during the shooting process by the mobile phone by obtaining the spatial position information collected by the gyroscope equipped with itself will be described in detail below with reference to FIGS. 3 to 6 . As shown in FIG. 3, the process includes the following steps 301-307.

Step 301 : The current frame image corresponding to the object to be photographed collected in real time is displayed on the screen of the mobile phone.

In this embodiment, the user uses the front or rear camera mounted on the mobile phone to shoot the object to be shot, such as using the front camera to take a selfie, or using the rear camera to shoot other people or objects. After the user turns on the camera function of the mobile phone, the mobile phone calls the camera to collect the optical information of the object to be photographed, and displays the corresponding preview image on the screen. The magic expression function is enabled by default on the mobile phone or the user turns on the magic expression function when the preview image is displayed on the mobile phone. , the corresponding magic emoji will be displayed in the preview screen of the mobile phone screen. As shown in FIG. 1 , when the object to be photographed is a rabbit ear, the special effect expression will be correspondingly displayed on the head position of the human face in the preview screen. Furthermore, when the user is satisfied with the current preview image and the overall preview effect of the magic expression, the user can capture the image by triggering the shooting function (eg, click the "shoot" control) to obtain a photo.

For another example, after the user turns on the video recording function of the mobile phone and starts recording, if the mobile phone has the magic expression function enabled by default or the user enables the magic expression function when displaying the preview screen on the mobile phone, the preview screen of the mobile phone screen will display the corresponding Magical emoji. Furthermore, the mobile phone records the display effect and dynamic change process of the magic expression in the recorded video.

In this embodiment, the image containing the magic expression displayed on the screen of the mobile phone is displayed frame by frame. Therefore, the process of displaying the magic expression in this solution is also displayed frame by frame. In some embodiments, during the process of displaying the current frame, the mobile phone determines the display parameters of the magic expression in the next frame by performing the following steps 302-305, and then the GPU shader renders the magic expression according to the display parameters to obtain the magic expression. The next frame of the expression is used for display.

Step 302: Determine the magic expression to be displayed.

In the process of displaying the current frame picture, the mobile phone determines the magic expression displayed in the current picture, and then determines the magic expression to be displayed. For example, in the case where it is determined that the currently selected and displayed magic expression among the candidate magic expressions is the rabbit ear, the mobile phone determines the rabbit ear as the magic expression to be displayed, that is, the rabbit needs to be displayed in the next frame of image. Ears (only the specific display parameters need to be determined through the following steps, so that the rabbit ears displayed in the next frame of image can correspond to the spatial location information of the mobile phone at the current moment).

Step 303: Obtain the quaternion collected and output by the gyroscope.

In this embodiment, the user changes the posture of the mobile phone by manipulating hand movements during the process of shooting with the mobile phone, such as controlling the mobile phone to pan, tilt, rotate, etc., so that the mobile phone collects its own spatial position information through the gyroscope assembled by itself. The display effect of the magic expression is controlled by using the spatial position information, which includes a quaternion.

In some embodiments, a magic watch SDK is pre-stored in the mobile phone, and the magic watch SDK not only records the target type corresponding to any magic expression used by the mobile phone to realize, but also records the corresponding records calculated by the above-mentioned at least one kind of spatial position information The corresponding algorithm required to realize any of the magic expressions is recorded with the preset relation table in the embodiment shown in FIG. 2 . Or, in the case where the mobile phone displays the magic expression through its own application with the camera calling function (such as mobile phone native shooting application, third-party shooting application, third-party multimedia application, etc.), the above magic watch SDK or the corresponding relationship is pre-stored in the In the server corresponding to the application, the mobile phone obtains the above magic watch SDK or the preset relationship table from the server, and then executes the subsequent steps.

In some embodiments, after determining the magic expression to be displayed, the mobile phone calls the magic watch SDK or determines the spatial location information corresponding to the magic expression to be displayed according to the preset relationship table. It is worth noting that the "several spatial position information corresponding to any magic expression used for realization" recorded in the magic watch SDK is only the name of the spatial position information, not its specific value, that is, the aforementioned target type. For example, the "quaternion" needs to be used for the realization of the rabbit ear special effect, so after the mobile phone determines that the target special effect object is the rabbit ear, it determines the need to obtain the current quaternion of the mobile phone by querying the magic watch SDK, and then from the mobile phone's gyro The instrument obtains the specific value of the quaternion it has collected.

In this embodiment, taking the magic expression as an example of a separation color block for realizing an RGB separation effect, the mobile phone determines the spatial position information, such as a quaternion, corresponding to the RGB separation effect according to the above preset relationship table. In the case where the spatial position information corresponding to the RGB separation effect is determined to be a quaternion by the above method, it indicates that the separation color block corresponding to the RGB separation effect needs to be displayed in the next frame of picture, and then the quaternion needs to be used. A data to obtain the display parameters of the separated color block.

In some embodiments, the mobile phone (specifically, the CPU of the mobile phone, the same below) extracts the quaternion from the spatial position information collected by the gyroscope equipped with itself; or, the mobile phone requests the gyroscope for the quaternion collected by the acquirer arity.

Step 304: Determine the rotation angle corresponding to the quaternion.

In this embodiment, Euler angles are used to represent actions such as movement or rotation in the process of changing the spatial position of the mobile phone. In some embodiments, the quaternion is directly converted into Euler angles in the Cartesian coordinate system, or a rotation matrix for characterizing the orientation change of the mobile phone is constructed, and the quaternion is converted into the corresponding Euler based on the rotation matrix horn. In the embodiment of the present application, the timing of constructing the rotation matrix is not specifically limited; for example, it is constructed when the rotation angle corresponding to the quaternion is determined, or it is constructed in advance and stored in the mobile phone, and when the rotation angle corresponding to the quaternion is determined, , you can directly obtain the stored rotation matrix.

In some embodiments, the rotation angle corresponding to the change in the attitude of the mobile phone is determined in various ways according to the above Euler angles. In some embodiments, the above-mentioned Euler angle obtained by calculation is directly determined as the rotation angle, so as to simplify the way of determining the rotation angle and speed up the calculation speed of the display parameters. Or, in some embodiments, the calculated Euler angles are cached in a parameter queue, and the parameter queue also caches several historical Euler angles corresponding to several historical frames before the current frame, so that the queue The average value of each Euler angle is determined as the rotation angle, so as to avoid jumps in the display process of the magic expression, thereby presenting a relatively stable display effect of the magic expression.

Wherein, the historical Euler angle in the above parameter queue is the Euler angle corresponding to the continuous picture frame. The pull angle constitutes a parameter queue, and accordingly, the average value of the n Euler angles in the queue is determined as the rotation angle corresponding to the current frame. It can be understood that the larger the above n is, the smoother the target special effect object in the picture is. Of course, in the case that the above n is sufficiently small (for example, n≤10), the maximum value among the n Euler angles in the parameter queue is determined as the rotation angle. Alternatively, the average or maximum value of the preset multiple of the current Euler angle and the preset multiples of the respective historical Euler angles is used as the rotation angle, and details are not repeated here.

Step 305: Determine the display parameters of the magic expression in the next frame of picture.

In some embodiments, the presentation parameters can be determined in a number of ways based on the above-mentioned rotation angles. In some embodiments, the mobile phone calculates display parameters such as blur intensity according to the above rotation angle according to the preset algorithm recorded in the Magic Watch SDK. In some embodiments, the mobile phone calculates the corresponding display parameters after multiplying the above-mentioned rotation angle by a coefficient x1 greater than 1, so as to ensure that the corresponding display parameters can still be calculated even when the rotation angle is small, so as to ensure the correct display parameters. Sensitivity of phone control and magic emoji display. Alternatively, after multiplying the above rotation angle by a coefficient x2 less than 1, the corresponding display parameters are calculated to ensure that when the rotation angle is large, the display parameters are too large to prevent the target special effect object from jumping in the screen. The above-mentioned coefficients x1 and x2 are empirical values preset and verified according to actual conditions such as the type of image acquisition device, the size of the display screen, and the type of the target special effect object, which are not limited in this embodiment of the present disclosure.

In some embodiments, the above-mentioned spatial position information includes the rotation angle, and the target special effect object has a preset standard display angle. At this time, the mobile phone determines the deflection angle of the target special effect object based on the rotation angle, and based on the deflection angle, the standard display angle is determined. Rotation transformation is performed, and the rotated angle obtained by transformation is determined as the display parameter of the target special effect object. As shown in Figure 4, assuming that the standard display angle of the rabbit's ear is 0°, the rabbit's ear will appear as shown in Figure 4(a) in a normal state with a vertical center line (parallel to the longitudinal center axis of the mobile phone screen). Display state. When the user changes the spatial position of the mobile phone and causes the mobile phone to rotate to the left (the rotation angle is generated), the component of the rotation angle along the central axis of the mobile phone screen also changes, that is, the deflection angle (assumed to be 30°) is generated. At this time, the mobile phone is equivalent to It rotates 30° to the left along the longitudinal center axis of the mobile phone screen, and then the user's selfie avatar on the mobile phone screen is rotated 30° to the right relative to the mobile phone screen. At this time, the mobile phone rotates and transforms the standard display angle according to the rotation angle, that is, adding 30° to the standard display angle, so that the rotated angle of the rabbit ear (relative to the rotation angle of the mobile phone screen) is 30° to the right, so that after the mobile phone is rotated The rabbit ears are still displayed above the face, keeping the relative positional relationship with the face unchanged, as shown in Figure 4(b). It can be seen that this method can ensure that the display effect of the target special effect object matches the relative positional relationship between the screen content in the screen captured by the mobile phone, and the display of the target special effect object will not be chaotic due to the rotation of the mobile phone.

Step 306: The CPU provides the display parameters to the GPU shader for image rendering to obtain the next frame of image.

Step 307: Display the next frame of image containing the magic expression.

In this embodiment, after the above-mentioned display parameters are determined, they are provided to the GPU shader corresponding to the screen of the mobile phone to render the next frame of picture, and then the screen of the mobile phone displays the next frame of picture after rendering, and the picture contains according to The magic expression rendered by the above display parameters, the display effect of the magic expression corresponds to the spatial posture of the mobile phone at the current moment. Wherein, the specific process of rendering and displaying the next frame of picture can refer to the description in the related art, and details are not repeated here.

The following description will be given by taking the magic expression as the separation color block corresponding to the RGB separation effect as an example, and the display effect of the separation color block can be referred to as shown in FIG. 5 and FIG. 6 .

Figure 5(a) shows a schematic diagram of the preview screen effect of the mobile phone shooting interface. When the user turns on the magic expression function in the interface and selects the RGB separation effect as the special effect, when the user controls the mobile phone to move from When rotated from right to left, the corresponding RGB separation effect is shown in Figure 5(b). Among them, a plurality of separate color blocks such as 501 and 502 are displayed. At this time, the display parameters corresponding to the separate color blocks are the number of separate color blocks, and also the width, blur intensity, RGB channel color value, and color saturation of each separate color block. degree, etc., which is not limited in the present disclosure. Similarly, as shown in Figure 6(a), when the user controls the mobile phone to rotate from top to bottom along its horizontal central axis, the corresponding RGB separation effect is shown in Figure 6(b), which will not be repeated.

As can be seen from Figure 5(a)-Figure 6(b), when the user controls the rotation of the mobile phone through hand movements, the display effect of the separated color blocks is correspondingly controlled, and then the RGB separation effect is controlled to be displayed in the next frame, thereby realizing Control the effect of RGB separation effects through the gesture pair of the mobile phone.

Corresponding to the foregoing embodiments of the special effect display method, the present disclosure also proposes an embodiment of a special effect display device.

FIG. 7 is a schematic block diagram of a special effect display device according to an embodiment of the present disclosure. The special effect display device shown in this embodiment is suitable for image acquisition equipment, the image acquisition equipment has the functions of image acquisition and special effect object display, and the image acquisition equipment includes but is not limited to mobile phones, tablet computers, wearable devices, personal computers, etc. Electronic equipment.

As shown in Figure 7, the special effect display device includes:

The information acquisition unit 701 is configured to acquire the spatial position information of the image acquisition device at the current moment;

A parameter determination unit 702, configured to determine display parameters of a target special effect object based on the spatial position information, where the target special effect object is used to process a picture captured by the image capture device;

The picture rendering unit 703 is configured to render the picture based on the display parameters, and display a special effect picture obtained by rendering, and the special effect picture includes the target special effect object.

In some embodiments, the spatial position information includes at least one of the following: acceleration, rotation angle, and quaternion.

In some embodiments, the information obtaining unit 701 is further configured to:

determining a target type, where the target type is the type of spatial position information required to display the target special effect object;

Based on the target type, the spatial location information is determined.

In some embodiments, the information obtaining unit 701 is further configured to:

The target type is determined by querying a preset relationship table, and the preset relationship table is used to record the corresponding relationship between the special effect object and the target type.

In some embodiments, the information obtaining unit 701 is further configured to:

Identifying the display object in the picture collected by the image acquisition device, and determining the candidate special effect object that matches the display object as the target special effect object; or,

In response to the effect selection instruction for the candidate effect object, the selected target effect object is determined.

In some embodiments, the spatial location information includes a quaternion, and the parameter determination unit 702 is further configured to:

converting the quaternion to Euler angles based on a three-dimensional rotation matrix;

Determine the display parameter of the target special effect object based on the Euler angle, or determine the display parameter of the target special effect object based on the product of the Euler angle and a preset multiple, where the preset multiple is not equal to A positive number of 1.

In some embodiments, the display parameters of the target special effect object are determined periodically, and the apparatus further includes:

The first history determination unit 704 is configured to obtain at least one historical product result determined in an adjacent historical period, the adjacent historical period includes at least one period that is located before the current period and is continuous, and determines that the product result is the same as the The maximum value or average value of at least one historical product result, and the maximum value or average value is determined as the display parameter of the target special effect object.

In some embodiments, the display parameters of the target special effect object are determined periodically, and the apparatus further includes:

The second history determination unit 705 is configured to acquire at least one historical Euler angle determined in an adjacent historical period, where the adjacent historical period includes at least one continuous period before the current period, and determine that the Euler angle is the same as the The maximum value or average value of the at least one historical Euler angle, and the maximum value or average value is determined as the display parameter of the target special effect object.

In some embodiments, the spatial position information includes acceleration, and the parameter determination unit 702 is further configured to:

determining a special effect display area, where the special effect display area is used to display the target special effect object;

Based on the acceleration, a pixel characteristic value in the special effect display area is determined, the pixel characteristic value is determined as a display parameter of the target special effect object, and the pixel characteristic value is used to represent the characteristic of the corresponding pixel point.

In some embodiments, the pixel feature value includes at least one of the following:

Color value, color gradient, transparency, contrast.

In some embodiments, the spatial position information includes a rotation angle, the target special effect object has a preset standard display angle, and the parameter determination unit 702 is further configured to:

determining the deflection angle of the target special effect object based on the rotation angle;

Based on the deflection angle, the standard display angle is rotated and transformed, and the rotated and transformed angle is determined as the display parameter of the target special effect object.

In some embodiments, the display parameters of the target special effect object include blur strength and/or color saturation.

Embodiments of the present disclosure also provide an electronic device, including:

processor;

a memory for storing the processor-executable instructions;

Wherein, the processor is configured to execute the instruction to implement the special effect display method according to any one of the above embodiments.

Embodiments of the present disclosure further provide a storage medium, when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the special effect display method described in any of the foregoing embodiments.

Embodiments of the present disclosure further provide a computer program product, the computer program product being configured to execute the special effect display method described in any of the foregoing embodiments.

FIG. 8 is a schematic block diagram of an electronic device according to an embodiment of the present disclosure. For example, electronic device 800 is a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, and the like.

8, an electronic device 800 includes one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and communication component 818.

The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 includes one or more processors 820 to execute the instructions to complete all or part of the steps of the above-mentioned special effect display method. Additionally, processing component 802 includes one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 includes a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operation at electronic device 800 . Examples of such data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. Memory 804 is implemented by any type of volatile or non-volatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply assembly 806 provides power to various components of electronic device 800 . Power components 806 include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800 .

Multimedia component 808 includes a screen that provides an output interface between electronic device 800 and the user. In some embodiments, the screen includes a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen is implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor not only senses the boundaries of a touch or swipe action, but also detects the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera are used to receive external multimedia data. Each front and rear camera is a fixed optical lens system or has focal length and optical zoom capability.

Audio component 810 is configured to output and/or input audio signals. For example, audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 800 is in operating modes, such as calling mode, recording mode, and voice recognition mode. The received audio signal is further stored in memory 804 or transmitted via communication component 818 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, and the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of electronic device 800 . For example, the sensor assembly 814 is used to detect the open/closed state of the electronic device 800, the relative positioning of the components, such as the display and the keypad of the electronic device 800, the sensor assembly 814 is also used to detect the electronic device 800 or the electronic device Changes in the position of a component 800 , presence or absence of user contact with the electronic device 800 , orientation or acceleration/deceleration of the electronic device 800 and changes in the temperature of the electronic device 800 . Sensor assembly 814 includes a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 also includes a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 also includes an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The image acquisition component 816 is used to acquire image data of the subject to form an image about the subject, and then perform necessary processing on the image. The image acquisition component 816 includes a camera module, and the image sensor (Sensor) in the camera module senses the light from the subject through the lens, and provides the obtained photosensitive data to an image signal processor (ISP, Image Signal Processing), which is composed of The latter generates an image corresponding to the subject based on the photosensitive data. Wherein, the above-mentioned image sensor is a CMOS sensor or a CCD sensor, of course, it is also an infrared sensor, a depth sensor, etc.; the camera module is built in the electronic device 800, or the camera module is used as an external module of the electronic device 800; the above-mentioned The ISP is built into the camera module, or as a plug-in in the above electronic device (not in the camera module).

Communication component 818 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 is capable of accessing wireless networks based on communication standards, such as WiFi, carrier networks (eg, 2G, 3G, 4G, or 5G), or a combination thereof. In one exemplary embodiment, the communication component 818 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 818 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In one embodiment of the present disclosure, the electronic device 800 is implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmed gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components are implemented for implementing the above-mentioned special effect display method.

In an embodiment of the present disclosure, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the electronic device 800 to complete the above-mentioned special effect display method. For example, the non-transitory computer-readable storage medium is ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

All the embodiments of the present disclosure can be implemented independently or in combination with other embodiments, which are all regarded as the protection scope required by the present disclosure.

In some embodiments, the spatial position information includes at least one of the following: acceleration, rotation angle, and quaternion.

In some embodiments, the pixel feature value includes at least one of the following:

Color value, color gradient, transparency, contrast.

In some embodiments, the display parameters of the target special effect object include blur strength and/or color saturation.

Claims (29)

1. A special effect display method, comprising:

   Obtain the spatial location information of the image acquisition device at the current moment;

   determining the display parameters of the target special effect object based on the spatial position information, and the target special effect object is used to process the picture captured by the image capture device;

   The picture is rendered based on the display parameters, and the special effect picture obtained by rendering is displayed, and the special effect picture includes the target special effect object.
2. The method according to claim 1, wherein the acquiring the spatial position information of the image acquisition device at the current moment comprises:

   determining a target type, where the target type is the type of spatial position information required to display the target special effect object;

   Based on the target type, the spatial location information is determined.
3. The method according to claim 2, wherein the determining the target type comprises:

   The target type is determined by querying a preset relationship table, and the preset relationship table is used to record the corresponding relationship between the special effect object and the target type.
4. The method of claim 1, wherein the method further comprises:

   Identifying a display object in the picture, and determining a candidate special effect object that matches the display object as the target special effect object; or,

   In response to the effect selection instruction for the candidate effect object, the selected target effect object is determined.
5. The method according to claim 1, wherein the spatial position information comprises a quaternion, and the determining the display parameters of the target special effect object based on the spatial position information comprises:

   converting the quaternion to Euler angles based on a three-dimensional rotation matrix;

   Determine the display parameter of the target special effect object based on the Euler angle, or determine the display parameter of the target special effect object based on the product of the Euler angle and a preset multiple, where the preset multiple is not equal to A positive number of 1.
6. The method according to claim 5, wherein the display parameters of the target special effect object are determined periodically; the display parameters of the target special effect object are determined based on a product result of the Euler angle and a preset multiple, include:

   Obtain at least one historical product result determined in an adjacent historical period, where the adjacent historical period includes at least one continuous period before the current period, and determine the maximum or average value of the product result and the at least one historical product result value, and the maximum value or the average value is determined as the display parameter of the target special effect object.
7. The method according to claim 5, wherein the display parameters of the target special effect object are determined periodically; the determining the display parameters of the target special effect object based on the Euler angles comprises:

   Obtain at least one historical Euler angle determined in an adjacent historical period, where the adjacent historical period includes at least one continuous period before the current period, and determine the maximum value between the Euler angle and the at least one historical Euler angle value or average value, and the maximum value or average value is determined as the display parameter of the target special effect object.
8. The method according to claim 1, wherein the spatial position information includes acceleration, and the determining a display parameter of the target special effect object based on the spatial position information comprises:

   determining a special effect display area, where the special effect display area is used to display the target special effect object;

   Based on the acceleration, a pixel characteristic value in the special effect display area is determined, the pixel characteristic value is determined as a display parameter of the target special effect object, and the pixel characteristic value is used to represent the characteristic of the corresponding pixel point.
9. The method according to claim 1, wherein the spatial position information includes a rotation angle, the target special effect object has a preset standard display angle, and the display parameters of the target special effect object are determined based on the spatial position information, include:

   determining the deflection angle of the target special effect object based on the rotation angle;

   Based on the deflection angle, the standard display angle is rotated and transformed, and the rotated and transformed angle is determined as the display parameter of the target special effect object.
10. A special effect display device, comprising:

    an information acquisition unit, configured to acquire the spatial position information of the image acquisition device at the current moment;

    a parameter determination unit, configured to determine display parameters of a target special effect object based on the spatial position information, where the target special effect object is used to process a picture captured by the image capture device;

    A picture rendering unit, configured to render the picture based on the display parameters, and display a special effect picture obtained by rendering, and the special effect picture includes the target special effect object.
11. The apparatus according to claim 10, wherein the information acquisition unit is further configured to:

    determining a target type, where the target type is the type of spatial position information required to display the target special effect object;

    Based on the target type, the spatial location information is determined.
12. The apparatus of claim 11, wherein the information acquisition unit is further configured to:

    The target type is determined by querying a preset relationship table, and the preset relationship table is used to record the corresponding relationship between the special effect object and the target type.
13. The apparatus according to claim 10, wherein the information acquisition unit is further configured to:

    Identifying a display object in the picture, and determining a candidate special effect object that matches the display object as the target special effect object; or,

    In response to the effect selection instruction for the candidate effect object, the selected target effect object is determined.
14. The apparatus according to claim 10, wherein the spatial position information comprises a quaternion, and the parameter determination unit is further configured to:

    converting the quaternion to Euler angles based on a three-dimensional rotation matrix;

    Determine the display parameter of the target special effect object based on the Euler angle, or determine the display parameter of the target special effect object based on the product of the Euler angle and a preset multiple, where the preset multiple is not equal to A positive number of 1.
15. The apparatus according to claim 14, wherein the display parameters of the target special effect object are determined periodically, and the apparatus further comprises:

    The first history determination unit is configured to obtain at least one historical product result determined in an adjacent historical period, the adjacent historical period includes at least one period that is located before the current period and is continuous, and determines that the product result is the same as the at least one historical period. A maximum value or average value of a historical product result, and the maximum value or average value is determined as the display parameter of the target special effect object.
16. The apparatus according to claim 14, wherein the display parameters of the target special effect object are determined periodically, and the apparatus further comprises:

    The second history determination unit is configured to obtain at least one historical Euler angle determined in an adjacent historical period, the adjacent historical period includes at least one period that is located before the current period and is continuous, and determines that the Euler angle is related to the selected historical period. The maximum value or average value of the at least one historical Euler angle is determined, and the maximum value or average value is determined as the display parameter of the target special effect object.
17. The apparatus of claim 10, wherein the spatial position information includes acceleration, and the parameter determination unit is further configured to:

    determining a special effect display area, where the special effect display area is used to display the target special effect object;

    Based on the acceleration, a pixel characteristic value in the special effect display area is determined, the pixel characteristic value is determined as a display parameter of the target special effect object, and the pixel characteristic value is used to represent the characteristic of the corresponding pixel point.
18. The device according to claim 11, wherein the spatial position information includes a rotation angle, the target special effect object has a preset standard display angle, and the parameter determination unit is further configured to:

    determining the deflection angle of the target special effect object based on the rotation angle;

    Based on the deflection angle, the standard display angle is rotated and transformed, and the rotated and transformed angle is determined as the display parameter of the target special effect object.
19. An electronic device comprising:

    processor;

    a memory for storing the processor-executable instructions;

    Wherein, the processor is configured to execute the instructions to achieve the following steps:

    Obtain the spatial location information of the image acquisition device at the current moment;

    determining the display parameters of the target special effect object based on the spatial position information, and the target special effect object is used to process the picture captured by the image capture device;

    The picture is rendered based on the display parameters, and the special effect picture obtained by rendering is displayed, and the special effect picture includes the target special effect object.
20. 19. The electronic device of claim 19, wherein the processor is configured to execute the instructions to implement the steps of:

    determining a target type, where the target type is the type of spatial position information required to display the target special effect object;

    Based on the target type, the spatial location information is determined.
21. 21. The electronic device of claim 20, wherein the processor is configured to execute the instructions to implement the steps of:

    The target type is determined by querying a preset relationship table, and the preset relationship table is used to record the corresponding relationship between the special effect object and the target type.
22. 19. The electronic device of claim 19, wherein the processor is configured to execute the instructions to implement the steps of:

    Identifying a display object in the picture, and determining a candidate special effect object that matches the display object as the target special effect object; or,

    In response to the effect selection instruction for the candidate effect object, the selected target effect object is determined.
23. 19. The electronic device of claim 19, wherein the spatial location information includes a quaternion, and the processor is configured to execute the instructions to implement the steps of:

    converting the quaternion to Euler angles based on a three-dimensional rotation matrix;

    Determine the display parameter of the target special effect object based on the Euler angle, or determine the display parameter of the target special effect object based on the product of the Euler angle and a preset multiple, where the preset multiple is not equal to A positive number of 1.
24. The electronic device according to claim 23, wherein the display parameters of the target special effect object are determined periodically; the processor is configured to execute the instructions to achieve the following steps:

    Obtain at least one historical product result determined in an adjacent historical period, where the adjacent historical period includes at least one period that is located before the current period and is continuous, and determine the maximum or average value of the product result and the at least one historical product result value, and the maximum value or the average value is determined as the display parameter of the target special effect object.
25. The electronic device according to claim 23, wherein the display parameters of the target special effect object are determined periodically; the processor is configured to execute the instructions to achieve the following steps:

    Obtain at least one historical Euler angle determined in an adjacent historical period, where the adjacent historical period includes at least one continuous period before the current period, and determine the maximum value between the Euler angle and the at least one historical Euler angle value or average value, and the maximum value or average value is determined as the display parameter of the target special effect object.
26. 21. The electronic device of claim 20, wherein the spatial position information includes acceleration, and the processor is configured to execute the instructions to:

    determining a special effect display area, where the special effect display area is used to display the target special effect object;

    Based on the acceleration, a pixel characteristic value in the special effect display area is determined, the pixel characteristic value is determined as a display parameter of the target special effect object, and the pixel characteristic value is used to represent the characteristic of the corresponding pixel point.
27. The electronic device of claim 20, wherein the spatial position information includes a rotation angle, and the processor is configured to execute the instructions to implement the steps of:

    determining the deflection angle of the target special effect object based on the rotation angle;

    Based on the deflection angle, the standard display angle is rotated and transformed, and the rotated and transformed angle is determined as the display parameter of the target special effect object.
28. A computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can perform the following steps:

    Obtain the spatial location information of the image acquisition device at the current moment;

    determining the display parameters of the target special effect object based on the spatial position information, and the target special effect object is used to process the picture captured by the image capture device;

    The picture is rendered based on the display parameters, and the special effect picture obtained by rendering is displayed, and the special effect picture includes the target special effect object.
29. A computer program product, comprising a computer program/instruction, wherein the computer program/instruction implements the following steps when executed by a processor:

    Obtain the spatial location information of the image acquisition device at the current moment;

    determining the display parameters of the target special effect object based on the spatial position information, and the target special effect object is used to process the picture captured by the image capture device;

    The picture is rendered based on the display parameters, and the special effect picture obtained by rendering is displayed, and the special effect picture includes the target special effect object.

Images