Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, the invention provides an image special effect display method, which determines the position of a target point of each frame of image to which special effect elements are to be added according to the difference between the positions of the feature points of different frames of images, increases the number of the positions of the target points to which special effects can be added, and improves the continuity between the added special effect elements.
The invention provides an image special effect display device.
The invention provides an electronic device.
The invention provides a computer readable storage medium.
The embodiment of the first aspect of the invention provides an image special effect display method, which comprises the following steps:
acquiring multi-frame images acquired sequentially, and acquiring the positions of feature points obtained by identifying the feature points of the multi-frame images;
determining the position of a target point to which a special effect element is to be added for each frame of image according to the difference between the positions of the feature points of different frames of images;
adding special effect elements in each frame image according to the position of the target point in each frame image;
and displaying each frame of image added with the special effect elements.
Optionally, as a first possible implementation manner of the first aspect of the present invention, the determining, according to a difference between feature point positions of different frame images, a target point position to which a special effect element is to be added for each frame image includes:
and determining the position of a target point to which a special effect element is to be added in a subsequent frame of the two adjacent frames of images according to the difference between the positions of the feature points identified by the two adjacent frames of images.
Optionally, as a second possible implementation manner of the first aspect of the present invention, the determining, according to a difference between feature point positions obtained by identifying two adjacent frames of images, a position of a target point to which a special effect element is to be added in a subsequent frame of the two adjacent frames of images includes:
determining a first position of a feature point in a previous frame of image of two adjacent frames of images;
determining a second position of the feature point in the next frame of image after the two adjacent frames of images;
determining a connecting line between the first position and the second position in a subsequent frame of image;
and determining the position of the target point according to the connection line.
Optionally, as a third possible implementation manner of the first aspect of the present invention, the determining the position of the target point according to the connection line includes:
determining the number of target points according to the length of the connecting line;
determining the positions of the target points according to the positions of the connecting lines, wherein the intervals among the target points accord with preset intervals; the preset interval is determined according to the size of the special effect element.
Optionally, as a fourth possible implementation manner of the first aspect of the present invention, the method further includes:
and if the situation that the feature point does not exist in the previous frame image of the two adjacent frames of images is determined, adding the special effect element at the feature point position in the next frame image.
Optionally, as a fifth possible implementation manner of the first aspect of the present invention, before the acquiring the position obtained by performing feature point identification on each frame of image, the method further includes:
reading a value of an objective function every time a frame of image is acquired, wherein the objective function is used for indicating whether a feature point is obtained by identifying the currently acquired image or not;
and determining the characteristic points which are obtained by identifying the currently acquired image according to the value of the objective function.
Optionally, as a sixth possible implementation manner of the first aspect of the present invention, the method further includes:
and if it is determined that the currently acquired image is not identified to obtain the feature points according to the value of the objective function, determining that special effect elements do not need to be added to the currently acquired image.
The embodiment of the second aspect of the invention provides an image special effect display device, which comprises:
the acquisition module is used for acquiring a plurality of frames of images which are sequentially acquired and acquiring positions obtained by identifying feature points of each frame of image;
the first determining module is used for determining the position of a target point to be added with a special effect element for each frame of image according to the difference between positions obtained by carrying out feature point identification on different frames of images;
the processing module is used for adding special effect elements in each frame image according to the position of a target point in each frame image;
and the display module is used for displaying the sequence of the frames of images added with the special effect elements.
Optionally, as a first possible implementation manner of the second aspect of the present invention, the first determining module is configured to:
and determining the position of a target point to which a special effect element is to be added in a subsequent frame of the two adjacent frames of images according to the difference between the positions of the feature points identified by the two adjacent frames of images.
Optionally, as a second possible implementation manner of the second aspect of the present invention, the first determining module is specifically configured to:
determining a first position of a feature point in a previous frame of image of two adjacent frames of images;
determining a second position of the feature point in the next frame of image after the two adjacent frames of images;
determining a connecting line between the first position and the second position in a subsequent frame of image;
and determining the position of the target point according to the connection line.
Optionally, as a third possible implementation manner of the second aspect of the present invention, the first determining module is further configured to:
determining the number of target points according to the length of the connecting line;
determining the positions of the target points according to the positions of the connecting lines, wherein the intervals among the target points accord with preset intervals; the preset interval is determined according to the size of the special effect element.
Optionally, as a fourth possible implementation manner of the second aspect of the present invention, the apparatus further includes:
and the second determining module is used for adding the special effect element at the position of the characteristic point in the next frame image if the characteristic point does not exist in the previous frame image of the two adjacent frames of images.
Optionally, as a fifth possible implementation manner of the second aspect of the present invention, the apparatus further includes:
the reading module is used for reading the value of an objective function when a frame of image is acquired, wherein the objective function is used for indicating whether the currently acquired image is identified to obtain a characteristic point;
and the first identification module is used for determining the identified characteristic points of the currently acquired image according to the value of the objective function.
Optionally, as a sixth possible implementation manner of the second aspect of the present invention, the apparatus further includes:
and the second identification module is used for determining that special effect elements do not need to be added to the currently acquired image if the currently acquired image is determined not to be identified to obtain the characteristic points according to the value of the target function.
An embodiment of a third aspect of the present invention provides an electronic device, including: the image special effect display method comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the image special effect display method of the first aspect is realized.
An embodiment of a fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for displaying an image special effect according to the first aspect.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
the method comprises the steps of obtaining multi-frame images collected in sequence, obtaining feature point positions obtained by carrying out feature point identification on the multi-frame images, determining the position of a target point to be added with a special effect element for each frame of image according to the difference between the feature point positions of different frames of images, adding the special effect element into each frame of image according to the position of the target point in each frame of image, and displaying each frame of image added with the special effect element. According to the difference between the positions of the feature points of different frame images, the positions of the target points of the frame images to which special effect elements are to be added are determined, the number of the positions of the target points to which special effects can be added is increased, and the consistency between the added special effect elements is improved.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The following describes an image special effect display method, an image special effect display device and electronic equipment according to an embodiment of the invention with reference to the drawings.
Fig. 1 is a flowchart illustrating an image special effect displaying method according to an embodiment of the present invention.
As shown in fig. 1, the method comprises the steps of:
step 101, obtaining a plurality of frames of images collected sequentially, and obtaining the position of a feature point obtained by identifying the feature point of the plurality of frames of images.
Specifically, multi-frame images acquired sequentially are acquired, feature points in the images are identified, and positions of the identified feature points are determined, where the feature points are points with specific features, such as hands, feet, and the like of people in the acquired images, and the positions of the feature points in different frame images may be the same or different.
And step 102, determining the position of a target point to be added with a special effect element for each frame of image according to the difference between the positions of the feature points of different frames of images.
Specifically, the position of a target point to which a special effect element is to be added in a subsequent frame of two adjacent frames of images is determined according to the difference between the positions of the feature points identified by the two adjacent frames of images.
It should be understood that the target point position includes not only the feature point position identified in each frame of image, but also the position of the special effect element to be added, which is determined according to the difference between the feature point positions of different frames of images, and the determination method is described in detail in the following embodiments.
Step 103, adding special effect elements in each frame image according to the position of the target point in each frame image.
Specifically, according to the position of the target point in each frame image, a special effect element is added to the position of the target point in each frame image, so that the special effect element of the target point has coherence.
And 104, displaying each frame of image added with the special effect element.
Specifically, each frame of image added with the special effect element is displayed, so that the displayed special effect element has better consistency in playing and does not have a stagnant visual sense of incongruity.
In the image special effect displaying method of this embodiment, a plurality of frames of images acquired sequentially are acquired, feature point positions obtained by identifying feature points of the plurality of frames of images are acquired, a target point position to which a special effect element is to be added is determined for each frame of image according to a difference between feature point positions of different frames of images, a special effect element is added to each frame of image according to the target point position in each frame of image, and each frame of image to which the special effect element is added is displayed. According to the difference between the positions of the feature points of different frame images, the position of the target point of each frame image to which a special effect element is to be added is determined, the number of the positions of the target points to which the special effects can be added is increased, the consistency between the added special effect elements is improved, and the problems that in the prior art, when the feature points move fast, the positions of the feature points in adjacent frames are far away, the special effect elements are added to the recognized positions of the feature points, the special effect elements are not connected, and the display effect is poor are solved.
To clearly illustrate the previous embodiment, the present embodiment provides another image special effect displaying method, and fig. 2 is a schematic flow chart of the another image special effect displaying method provided by the embodiment of the present invention.
As shown in fig. 2, the method may include the steps of:
step 201, acquiring an image, and reading a value of the objective function every time one frame of image is acquired.
In an application scenario of the embodiment of the invention, a user dances according to a body feeling dance game played by a mobile terminal, the mobile terminal collects a dance video of the user in a dance process of the user, obtains a video frame, and identifies characteristic points of a human body in the frame.
Specifically, in the image acquisition process, the value of the objective function is read every time one frame of image is acquired, wherein the objective function is used for indicating whether the currently acquired image is identified to obtain the feature point.
The value of the objective function is determined according to the result of the feature point identification in the image, and if the feature point is identified in the image, the value of the objective function indicates that the feature point is identified and obtained in the currently acquired image, for example, the value is 1; if the feature point is not identified in the image, the value of the objective function indicates that the feature point is not identified in the currently acquired image, for example, the value is 0. The method for identifying the characteristic points in the image is a possible implementation manner, the device for acquiring the image can be a camera device capable of acquiring depth information, a three-dimensional model of the human body is constructed according to the acquired depth information, and each characteristic point of the human body is identified and obtained according to the characteristics of each part of the human body in the three-dimensional model. However, the method for identifying the feature points in the embodiment of the present invention is not limited.
Step 202, determining whether the acquired image is identified to obtain the feature points according to the value of the objective function, if so, executing step 204, and if not, executing step 203.
Specifically, according to the value of the objective function, if it is determined that the feature point is identified in the acquired image, step 204 is executed, and if it is determined that the feature point is not identified in the acquired image, step 203 is executed.
Step 203, determining that no special effect element needs to be added to the currently acquired image.
Specifically, if it is determined that the feature point is not identified in the currently acquired image, the special effect element does not need to be added to the currently acquired image.
And 204, acquiring the position of the feature point obtained by identifying the feature point of the current image.
Specifically, for determining the current image with the identified feature points, the feature point positions are identified to obtain a pixel range corresponding to the feature points, a pixel point corresponding to the center of the feature point is obtained, and the position of the feature point in the image is determined according to the distance between the pixel point and each edge of the image.
It should be noted that, in the embodiment of the present invention, a method for identifying positions of feature points is not limited.
Step 205, acquiring a plurality of frames of images collected sequentially.
Specifically, the acquired multi-frame images are acquired according to the acquisition sequence during image acquisition.
Step 206, determining whether a feature point exists in a previous image of two adjacent images, if so, executing step 208, and if not, executing step 207.
Specifically, according to the sequence of the collected multiple frames of images, whether a feature point exists in a previous frame of image in two adjacent frames of images is sequentially obtained, if the feature point exists, step 208 is executed, and if the feature point does not exist, step 207 is executed.
Step 207, adding special effect elements to the positions of the feature points in the next frame of image.
Specifically, if there is no feature point in the image of the previous frame in the two adjacent frames of images, a special effect element is added only to the feature point position in the image of the next frame.
And step 208, determining the position of a target point to be added with a special effect element in the next frame of the two adjacent frames of images according to the difference between the positions of the feature points obtained by the two adjacent frames of images.
Specifically, if a feature point in an image of a previous frame exists in two adjacent frames of images, a first position where the feature point in the image of the previous frame exists in the two adjacent frames of images is determined, a second position where the feature point in the image of a subsequent frame exists in the two adjacent frames of images is determined, a connecting line between the first position and the second position is determined in the image of the subsequent frame, and the number of special effect elements which can be added to the connecting line between the first position and the second position is determined according to the length of the connecting line and the size of the special effect elements which need to be added. The positions of the target points are determined according to the positions of the connecting lines and the preset intervals among the special effect elements, the intervals among the target points accord with the preset intervals, the preset intervals are determined according to the sizes of the special effect elements, the target points are added according to the connecting lines between the first positions where the feature points in the previous frame of image are located and the second positions where the feature points in the next frame of image are located, data of the target points between two adjacent frames are improved, the distribution of the target points is enabled to be more consistent, and the consistency of particles corresponding to the special effect elements after the special effect elements are added subsequently is improved.
Step 209, add special effect elements in each frame image according to the position of the target point in each frame image.
Specifically, as a possible implementation manner, special effect elements are added to each frame image according to the position of a target point in each frame image, particles are emitted to the position of the target point by using a particle emitter kewlparticle filter according to the form of a preset special effect element, for example, the special effect element can be in a star shape, a love shape or a flame shape, and the interest is increased by adding the special effect elements to the feature points.
And step 210, displaying each frame of image after the special effect elements are added.
In the embodiment of the present invention, a hand is taken as an example to explain, and fig. 3 is a schematic diagram illustrating a special effect element provided in the embodiment of the present invention, as shown in fig. 3, a left diagram illustrates particles emitted by a particle emitter, the particles are in a star shape, and a coherent special effect illustrated in a right diagram is formed according to the star-shaped particles emitted by the particle emitter. The arrow direction in the right image indicates the direction of the connecting line of the target point corresponding to the left-hand side determined according to each frame image, the special effect particles emitted by the particle emitter are dynamically added at the target point position according to the target point position determined in each frame image, the star special effect particles scattered all around are dynamically added at the target point position, and the complete special effect is obtained after each frame image is sequentially displayed, so that the continuity among the particles corresponding to the special effect stars is improved, and the effect is more natural and smooth.
In the image special effect display method of the embodiment of the invention, the feature point identification is carried out aiming at each collected frame image, the first position of the feature point in the previous frame image and the second position of the feature point in the next frame image in the two adjacent frame images are determined according to the difference between the feature point positions identified by the two connected frame images for the position corresponding to the image acquisition feature point of which the feature point is identified, the connecting line between the first position and the second position is determined in the next frame image, the number of the target points and the position of the target point are determined according to the connecting line, the number of the target points which can be added with the special effect element on the connecting line is increased, and the generated special effect is more coherent and natural.
In order to implement the above embodiments, the present invention further provides an image special effect display device.
Fig. 4 is a schematic structural diagram of an image special effect display device according to an embodiment of the present invention.
As shown in fig. 4, the apparatus includes: an acquisition module 41, a first determination module 42, a processing module 43 and a presentation module 44.
The obtaining module 41 is configured to obtain multiple frames of images collected sequentially and obtain positions obtained by performing feature point identification on each frame of image.
The first determining module 42 is configured to determine, for each frame image, a position of a target point to which a special effect element is to be added according to a difference between positions obtained by performing feature point identification on different frame images.
And the processing module 43 is configured to add a special effect element to each frame image according to the position of the target point in each frame image.
And a display module 44, configured to sequentially display the frames of images to which the special effect element is added.
It should be noted that the foregoing explanation of the method embodiment is also applicable to the apparatus of this embodiment, and is not repeated herein.
In the image special effect display device of this embodiment, a plurality of frames of images acquired in sequence are acquired, feature point positions obtained by identifying feature points of the plurality of frames of images are acquired, a target point position to which a special effect element is to be added is determined for each frame of image according to a difference between feature point positions of different frames of images, a special effect element is added to each frame of image according to the target point position in each frame of image, and each frame of image to which the special effect element is added is displayed. According to the difference between the positions of the feature points of different frame images, the positions of the target points of the frame images to which special effect elements are to be added are determined, the number of the positions of the target points to which special effects can be added is increased, and the consistency between the added special effect elements is improved.
Based on the foregoing embodiment, the embodiment of the present invention further provides a possible implementation manner of an image special effect display device, and fig. 5 is a schematic structural diagram of another image special effect display device provided in the embodiment of the present invention, where on the basis of the foregoing embodiment, the device further includes: a reading module 45, a first identification module 46, a second identification module 47 and a second determination module 48.
A reading module 45, configured to read a value of an objective function each time a frame of image is acquired, where the objective function is used to indicate whether a feature point is obtained by identifying a currently acquired image;
and a first identification module 46, configured to determine, according to a value of the objective function, that the feature point is identified in the currently acquired image.
The second identifying module 47 is configured to determine that a special effect element does not need to be added to the currently acquired image if it is determined according to the value of the objective function that the currently acquired image is not identified to obtain the feature point.
And a second determining module 48, configured to add a special effect element to a position of a feature point in a next frame of image if it is determined that there is no feature point in a previous frame of image of two adjacent frames of images.
Furthermore, as a possible implementation manner of the embodiment of the present invention, the first determining module 42 is specifically configured to:
and determining the position of a target point to which a special effect element is to be added in the next frame of the two adjacent frames of images according to the difference between the positions of the feature points identified by the two adjacent frames of images.
As another possible implementation manner, the first determining module 42 is further configured to:
determining a first position of a feature point in a previous frame of image of two adjacent frames of images;
determining a second position of the feature point in the next frame of image after the two adjacent frames of images;
determining a connection line between the first position and the second position in the subsequent frame image;
and determining the position of the target point according to the connecting line.
As another possible implementation manner, the first determining module 42 is further configured to:
determining the number of target points according to the length of the connecting line;
determining the positions of the target points according to the positions of the connecting lines, wherein the intervals among the target points accord with preset intervals; the preset interval is determined according to the size of the special effect element.
It should be noted that the foregoing explanation of the method embodiment is also applicable to the apparatus of this embodiment, and is not repeated herein.
In the image special effect display device according to the embodiment of the present invention, feature point identification is performed for each acquired frame image, for a position corresponding to an image acquisition feature point where the feature point is identified, according to a difference between feature point positions identified by two consecutive frames of images, a first position where the feature point in a previous frame image is located and a second position where the feature point in a subsequent frame image is located are determined in the two consecutive frames of images, a connection line between the first position and the second position is determined in the subsequent frame image, the number of target points and the position of the target point are determined according to the connection line, the number of target points to which special effect elements can be added on the connection line is increased, and a generated special effect is more coherent and natural.
In order to implement the above embodiments, the present invention further provides an electronic device, including: the image special effect display method comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the image special effect display method is realized.
An embodiment of the present invention further provides an electronic device, which includes the apparatus according to any of the foregoing embodiments.
Fig. 6 is a schematic structural diagram of an embodiment of an electronic device of the present invention, which may implement the processes of the method embodiments shown in fig. 1-2 of the present invention, and as shown in fig. 6, the electronic device may include: the electronic device comprises a shell 91, a processor 92, a memory 93, a circuit board 94 and a power supply circuit 95, wherein the circuit board 94 is arranged inside a space enclosed by the shell 91, and the processor 92 and the memory 93 are arranged on the circuit board 94; a power supply circuit 95 for supplying power to each circuit or device of the electronic apparatus; the memory 93 is used to store executable program code; the processor 92 executes a program corresponding to the executable program code by reading the executable program code stored in the memory 93, for executing the video generation method described in any one of the foregoing embodiments.
For the specific execution process of the above steps by the processor 92 and the steps further executed by the processor 92 by running the executable program code, reference may be made to the description of the method embodiment shown in fig. 1-2 of the present invention, which is not described herein again.
The electronic device exists in a variety of forms, including but not limited to:
(1) a mobile communication device: such devices are characterized by mobile communications capabilities and are primarily targeted at providing voice, data communications. Such terminals include: smart phones (e.g., iphones), multimedia phones, functional phones, and low-end phones, among others.
(2) Ultra mobile personal computer device: the equipment belongs to the category of personal computers, has calculation and processing functions and generally has the characteristic of mobile internet access. Such terminals include: PDA, MID, and UMPC devices, etc., such as ipads.
(3) A portable entertainment device: such devices can display and play multimedia content. This type of device comprises: audio, video players (e.g., ipods), handheld game consoles, electronic books, and smart toys and portable car navigation devices.
(4) A server: the device for providing the computing service comprises a processor, a hard disk, a memory, a system bus and the like, and the server is similar to a general computer architecture, but has higher requirements on processing capacity, stability, reliability, safety, expandability, manageability and the like because of the need of providing high-reliability service.
(5) And other electronic equipment with data interaction function.
In order to implement the foregoing embodiments, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the image special effect displaying method according to the foregoing method embodiments.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.