CN114564612A - Image matting effect display method and device, storage medium and electronic equipment - Google Patents

Image matting effect display method and device, storage medium and electronic equipment Download PDF

Info

Publication number
CN114564612A
CN114564612A CN202210089352.3A CN202210089352A CN114564612A CN 114564612 A CN114564612 A CN 114564612A CN 202210089352 A CN202210089352 A CN 202210089352A CN 114564612 A CN114564612 A CN 114564612A
Authority
CN
China
Prior art keywords
image
matting
keying
target
effect
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210089352.3A
Other languages
Chinese (zh)
Inventor
马银建
江欢
郑乃光
洪嘉慧
任振普
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Dajia Internet Information Technology Co Ltd
Original Assignee
Beijing Dajia Internet Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dajia Internet Information Technology Co Ltd filed Critical Beijing Dajia Internet Information Technology Co Ltd
Priority to CN202210089352.3A priority Critical patent/CN114564612A/en
Publication of CN114564612A publication Critical patent/CN114564612A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73Querying
    • G06F16/732Query formulation
    • G06F16/7335Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/74Browsing; Visualisation therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/7867Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/24Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Library & Information Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

The disclosure relates to a keying effect display method and device, a storage medium and an electronic device. The method comprises the steps that under the condition that a keying effect display instruction for a target video is obtained, a keying effect of a first image is displayed, wherein the first image is a first keyed image in the target video and responds to the keying instruction; displaying the keying effect of the third image under the condition that the second image does not exist; the second image is an image with a timestamp after the first image in the target video and which is scratched in response to the target scratching instruction, and the third image is an image with a timestamp after the first image and which is not adjacent to the first image in the target video. This disclosed embodiment can interval some image frames, carries out the demonstration of keying effect, and this kind of demonstration compares in frame-by-frame demonstration keying effect, can demonstrate the video and key the look of looking at fast.

Description

Image matting effect display method and device, storage medium and electronic equipment
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for displaying a keying effect, a storage medium, and an electronic device.
Background
With the development of image processing technology, more and more applications are obtained for performing image matting processing on videos. In the related technology, a matting instruction can be triggered for a certain frame in a video, then corresponding matting is also performed for other frames behind the frame in the video, and after the matting of each frame in the video is completed, a matting effect for the video is displayed. When the keying processing is carried out, the user can be reminded to wait in a picture prompting mode, however, the display mode of the video keying effect needs the user to wait for a long time, the user experience is reduced, and the viscosity of the user is influenced.
Disclosure of Invention
In order to solve at least one technical problem, the present disclosure provides a keying effect display method, device, storage medium and electronic device, and the technical scheme of the present disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a keying effect display method, including:
under the condition that a keying effect display instruction for a target video is obtained, displaying a keying effect of a first image, wherein the first image is a first keyed image in the target video in response to the target keying instruction;
displaying the keying effect of the third image under the condition that the second image does not exist;
wherein the second image is an image in the target video with a timestamp located after the first image and being scratched in response to the target scratching instruction,
the third image is an image of the target video, of which the timestamp is located after the first image and is not adjacent to the first image.
In some possible embodiments, after displaying the keying effect of the third image, the method further comprises:
and displaying the keying effect of a fourth image, wherein the fourth image is an image of the target video, the timestamp of which is positioned behind the third image and is not adjacent to the third image.
In some possible embodiments, the method further comprises:
and under the condition that at least one second image exists, sequentially displaying the keying effect of the second images according to the sequence of the second images in the target video.
In some possible embodiments, after the sequentially displaying the keying effect of the second image, the method further comprises:
displaying a keying effect of a fifth image under the condition that an image which is not keyed to the target keying instruction exists in the target video;
the fifth image is an image, of the target video, of which the timestamp is positioned after the target second image and is not adjacent to the target second image;
the target second image is the second image with the largest time stamp.
In some possible embodiments, after the displaying the keying effect of the fifth image, the method further comprises:
and displaying the keying effect of a sixth image, wherein the sixth image is an image of the target video, the timestamp of which is positioned behind the fifth image and is not adjacent to the fifth image.
In some possible embodiments, the method further comprises:
when the keying effect of the image which is already keyed is displayed, if an image which is not keyed according to the target keying instruction exists in the target video, video keying processing is performed in parallel;
the video matting processing comprises:
determining at least one image to be scratched in the target video according to the image which is scratched in response to the target scratching instruction in the target video;
and carrying out image matting aiming at the target image matting instruction on the at least one image to be subjected to image matting to obtain the image matting effect of the image to be subjected to image matting.
In some possible embodiments, the matting the at least one image to be scratched with respect to the target scratching instruction to obtain a scratching effect of the image to be scratched, including:
for each image to be scratched, determining a reference image, wherein the reference image is an image which is closest to and before a timestamp of the image to be scratched and is already scratched in response to the target scratching instruction in the target video;
and tracking the image matting effect corresponding to the reference image to obtain the image matting effect of the image to be subjected to image matting.
In some possible embodiments, the method further comprises:
and when a new matting instruction is not received after the preset time is exceeded and an image which is not subjected to matting aiming at the object matting instruction exists in the object video, performing the video matting processing.
In some possible embodiments, the method further comprises:
after any image in the target video is subjected to image matting aiming at the target image matting instruction, the image matting effect of any image is stored.
In some possible embodiments, the method further comprises:
and under the condition that an image which is not subjected to image matting aiming at the object image matting instruction exists in the object video, displaying prompt information, wherein the prompt information is used for prompting that the object video does not complete all image matting and the image matting effect of the object video displays that the image matting effect of the object video exists in a blocked condition.
According to a second aspect of the embodiments of the present disclosure, there is provided a keying effect display device, comprising:
the display device comprises a first display module and a second display module, wherein the first display module is configured to execute the step of displaying the keying effect of a first image under the condition that the keying effect display instruction for a target video is obtained, and the first image is a first keyed image in the target video in response to the target keying instruction;
a second display module configured to perform a matting effect of displaying a third image in the absence of the second image;
wherein the second image is an image in the target video with a timestamp located after the first image and being scratched in response to the target scratching instruction,
the third image is an image of the target video, of which the timestamp is located after the first image and is not adjacent to the first image.
In some possible embodiments, the second display module is configured to perform:
and displaying the keying effect of a fourth image, wherein the fourth image is an image of the target video, the timestamp of which is positioned behind the third image and is not adjacent to the third image.
In some possible embodiments, the second display module is configured to perform:
and under the condition that at least one second image exists, sequentially displaying the keying effect of the second images according to the sequence of the second images in the target video.
In some possible embodiments, the second display module is configured to perform:
displaying a keying effect of a fifth image under the condition that an image which is not keyed to the target keying instruction exists in the target video;
the fifth image is an image, of the target video, of which the timestamp is positioned after the target second image and is not adjacent to the target second image;
the target second image is the second image with the largest time stamp.
In some possible embodiments, the second display module is configured to perform:
and displaying the keying effect of a sixth image, wherein the sixth image is an image of the target video, the timestamp of which is positioned behind the fifth image and is not adjacent to the fifth image.
In some possible embodiments, the apparatus further comprises a matting module configured to perform:
when the keying effect of the image which is already keyed is displayed, if an image which is not keyed according to the target keying instruction exists in the target video, video keying processing is performed in parallel;
the video matting processing comprises:
determining at least one image to be scratched in the target video according to the image which is scratched in response to the target scratching instruction in the target video;
and carrying out image matting aiming at the target image matting instruction on the at least one image to be subjected to image matting to obtain the image matting effect of the image to be subjected to image matting.
In some possible embodiments, the matting module is configured to perform determining, for each image to be scratched, a reference image which is an image in the target video that has been scratched in response to the target scratching instruction and is closest to and before a timestamp of the image to be scratched;
and tracking the image matting effect corresponding to the reference image to obtain the image matting effect of the image to be subjected to image matting.
In some possible embodiments, the matting module is configured to perform
And when a new image matting instruction is not received beyond a preset time and an image which is not subjected to image matting aiming at the image matting instruction exists in the target video, carrying out the image matting processing.
In some possible embodiments, the matting module is configured to perform matting for any image in the target video after the matting for the target matting instruction is performed on the any image, and store a matting effect of the any image.
In some possible embodiments, the second display module is configured to perform:
and under the condition that an image which is not subjected to image matting aiming at the object image matting instruction exists in the object video, displaying prompt information, wherein the prompt information is used for prompting that the object video does not complete all image matting and the image matting effect of the object video displays that the image matting effect of the object video exists in a blocked condition.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method of any one of the first aspect as described above.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions, when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of the first aspects of the embodiments of the present disclosure.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising a computer program, the computer program being stored in a readable storage medium, from which at least one processor of a computer device reads and executes the computer program, causing the computer device to perform the method of any one of the first aspects of embodiments of the present disclosure.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
the keying effect display method provided by the embodiment of the disclosure can display the keying effect of a first image under the condition that a keying effect display instruction for a target video is obtained, wherein the first image is a first keyed image in the target video in response to the keying instruction; displaying the keying effect of the third image under the condition that the second image does not exist; wherein the second image is an image of the target video with a timestamp after the first image and which is scratched in response to the target scratching instruction, and the third image is an image of the target video with a timestamp after the first image and which is not adjacent to the first image. This disclosed embodiment can some image frames of interval, carry out the demonstration of keying effect, this kind of demonstration is compared in frame-by-frame demonstration keying effect, can demonstrate the video keying panorama fast, the image quantity of keying is less, consequently, display speed is faster, the card pause condition is lighter, need not carry out keying for a long time and wait, can respond to keying effect display instruction and show keying effect immediately, show reduction user latency, promote keying effect display speed, promote user's viscosity.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a schematic diagram of an implementation environment of a keying effect display method according to an exemplary embodiment;
FIG. 2 is a flow diagram illustrating a keying effect display method according to an exemplary embodiment;
FIG. 3 is a schematic diagram illustrating tracking according to an exemplary embodiment;
FIG. 4 is a schematic diagram of a matting interface shown in accordance with an exemplary embodiment;
FIG. 5 is a block diagram illustrating a keying effect display device according to an exemplary embodiment;
FIG. 6 is a block diagram illustrating an electronic device in accordance with an example embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Various data such as user information and images related to the present disclosure are information authorized by a user or sufficiently authorized by each party.
The keying is generally performed by using an artificial intelligence algorithm, relatively more computing resources and storage resources are consumed, especially for video keying, the consumption of the resources may cause a display card with a video keying effect to pause or wait for a long time, especially under the condition of a lower end of the device, the defect can be more prominent, in order to provide the capability of playing the keying effect of the video in real time, the waiting time of a user is reduced, and the user experience is improved.
Fig. 1 is a schematic diagram of an implementation environment of a keying effect display method according to an exemplary embodiment. Taking the electronic device as an example provided as a terminal, referring to fig. 1, the implementation environment specifically includes: a terminal 101 and a server 102.
The terminal 101 may be at least one of a smartphone, a smart watch, a desktop computer, a laptop portable computer, and the like. An application program providing a video matting service and a video matting effect playing service can be installed and run on the terminal 101, and a user can log in the application program through the terminal 101 to obtain the video matting service and the video matting effect playing service provided by the application program. The terminal 101 may be generally referred to as one of a plurality of terminals, and the embodiment is only illustrated by the terminal 101. Those skilled in the art will appreciate that the number of terminals described above may be greater or fewer. For example, the number of the terminals may be only a few, or the number of the terminals may be several tens or hundreds, or more, and the number of the terminals and the type of the device are not limited in the embodiments of the present disclosure.
The server 102 may be at least one of a server, a plurality of servers, a cloud computing platform, and a virtualization center. The server 102 may be connected to the terminal 101 and other terminals through a wireless network or a wired network, and the server 102 may receive a relevant request sent by the terminal 101, and provide a video matting service and a video matting effect playing service in cooperation with the terminal 101. Optionally, the number of the servers may be more or less, and the embodiment of the disclosure does not limit this. Of course, the server 102 may also include other functional servers to provide more comprehensive and diverse services. Fig. 2 is a flowchart illustrating a keying effect display method according to an exemplary embodiment, and as shown in fig. 2, the method includes at least the following steps S10-S30.
In step S10, when a matting effect display instruction for a target video is acquired, a matting effect of a first image is displayed, the first image being a first image to be scratched in response to the target matting instruction in the target video.
The disclosed embodiment does not limit the scene of displaying the keying effect of the target video, and the keying effect of the target video can be displayed in scenes such as video clips and video editing. The disclosed embodiment does not limit the source of the target video, such as from a gallery local to the terminal 101, from a video captured by a camera of the terminal 101, or from a network. The target video in the embodiment of the present disclosure may be a complete video or a video segment in a video.
In the embodiment of the present disclosure, a certain frame of a target video may be subjected to matting, and an image after the certain frame in the target video may also be subjected to matting based on the same principle, and an image corresponding to the certain frame is the first image above. For example, if a kitten in a target video "kitten drinks water" is scratched to a kitten in an image of a small bowl with a head lowered, an image of water in the small bowl drunk by the kitten and a kitten in an image of a water head raised drunk by the kitten can be scratched.
After the first image is subjected to image matting, under the condition that an image matting effect display instruction for the target video is obtained, an image matting result can be displayed, namely the image matting effect of the first image is displayed.
In step S20, in the case where the second image does not exist, the matting effect of the third image is displayed; the second image is an image with a timestamp after the first image in the target video and which is scratched in response to the target scratching instruction, and the third image is an image with a timestamp after the first image and which is not adjacent to the first image in the target video.
The second image in the embodiment of the present disclosure refers to an image in the target video, of which the time stamp is located after the first image and which is scratched in response to the target scratching instruction. For example, the target video includes 10 frames of images, and the target matting instruction performs matting on the beauty in the 2 nd frame of image, that is, the image corresponding to frame2 is the first image in step S10 in the above 10 frames of images, and the beauty in the images of frames 3-10 should also be matting in response to the above target matting instruction. If a girl-matte image is present in frame3-10, then this girl-matte image is the second image, and if a girl-matte image is not present in frame3-10, then this second image is not present.
In fact, if the matting effect for the above-described object matting instruction is displayed for the first time, then there must not be a second image, since no matting process for the object matting instruction has been performed for frame3-10 before, in which case the matting effect for a third image can be displayed. Still following the above example, the third image is an image in the target video whose timestamp is located after the first image and is not adjacent to the first image, that is, the third image may be any image of frames 4-10, after the keying effect of the first image is displayed, the keying effect of the third image is displayed, and the first image and the third image are not adjacent, so that the keying display is a frame skipping display.
Of course, after the keying effect of the third image is displayed, the keying effect of a fourth image may be continuously displayed, where the fourth image is an image of the target video whose timestamp is located after the third image and is not adjacent to the third image. Assuming that the third image is frame4, the fourth image may be any of frame 5-10. By analogy, for a 10-frame image, only frame2, frame4, frame8, and frame10 may be displayed in step S20.
That is to say, can some image frames of interval, carry out the demonstration of keying the image effect, this kind of demonstration is compared in frame-by-frame demonstration keying the image effect, can demonstrate the video and key the image panorama fast, the image quantity of keying the image is less, consequently, display speed is faster, the card pause condition is lighter, need not carry out keying for a long time and wait, can respond to keying the image effect display instruction and demonstrate keying the image effect immediately, show reduction user latency, promote keying the image effect display speed, promote user's viscosity.
In step S30, when at least one second image exists, the matting effect of the second images is sequentially displayed in the order of the second images in the target video.
If the over-keying effect display instruction is triggered after the object keying instruction is obtained, step S20 is executed at least once, that is, at least some images exist that have been keyed in response to the object keying instruction, and these images may be referred to as second images in the embodiment of the present disclosure, that is, the second images are the images that have been keyed in response to the object keying instruction and whose timestamps are behind the first images in the object video. Taking the example of frame4, frame6, and frame8 displayed in step S20, frame4, frame6, and frame8 all belong to the second image. In this case, if the matting effect display instruction is triggered again, steps S10 and S30 can be executed to directly display the matting effects of frames 2, 4, 6, and 8 without re-matting frames 4, 6, and 8, significantly improving the display speed of the matting effect.
In one embodiment, after the sequentially displaying the keying effect of the second image, the method further includes: displaying a keying effect of a fifth image under the condition that an image which is not keyed according to the target keying instruction exists in the target video; the fifth image is an image of the target video, whose timestamp is located after the target second image and is not adjacent to the target second image; the target second image is the second image with the largest time stamp. Taking the foregoing as an example, after the matting effects of frame2, frame4, frame6 and frame8 are sequentially displayed, there are images that have not been subjected to matting for the above-mentioned target matting instruction, such as frame9 and frame10, in which case the same frame skipping concept as step S20 can be employed to increase the display speed, such as the matting effect of frame10 can be displayed.
Of course, if there is an image after the fifth image that has not been subjected to the matting processing for the object matting instruction, frame skipping display may also be performed continuously, and specifically, a matting effect of a sixth image may be displayed, where the sixth image is an image in the object video whose timestamp is after the fifth image and is not adjacent to the fifth image. By analogy, the concept of the skip frame display is also described in the foregoing, and is not described herein.
That is, in the case where there is a frame that has already been scratched, the scratching effect is directly displayed; if meet and show the condition that does not have the frame of scratching, in order to promote the display speed of scratching the effect, this disclosed embodiment jumps the frame and shows to make the display frame to the scratching effect of target video more smooth, promote the picture smoothness degree, reduce the picture card pause, promote the display speed of scratching the effect, no longer need carry out the user and wait, thereby show promotion user experience.
In the embodiment of the disclosure, concurrent resources or idle resources can be fully utilized to perform image matting processing. In one embodiment, when the matting effect of the image that has been subjected to matting is displayed, if an image that has not been subjected to matting with respect to the target matting instruction is still present in the target video, the video matting processing may be performed in parallel. Through when the effect of scratching is shown, if still there is the image that is not carried out the scratching to above-mentioned target scratching instruction in the above-mentioned target video, then carry out the mode that the video was scratched and is handled in parallel, utilize calculation resource and time neutral to carry out the operation of scratching with higher speed better, accomplish the scratching as early as possible, reduce the broadcast card of the effect of scratching and pause.
Specifically, the video matting processing includes: determining at least one image to be scratched in the target video according to the image which is scratched in response to the target scratching instruction in the target video; and carrying out image matting aiming at the target image matting instruction on the at least one image to be subjected to image matting to obtain the image matting effect of the image to be subjected to image matting.
The embodiment of the present disclosure does not limit the determination method of the image to be scratched, for example, a target video has 10 frames in total, and is played from the 1 st frame during playing, and if the 1 st frame is scratched, the effect is obtained, and the played image is played, and the user sees the scratching effect, and the next 2-4 frames may not reach the scratching calculation due to insufficient calculation resources, directly jump to the 5 th frame to scratch, and show the effect of the 5 th frame, so the seen effect is the frame jump scratching effect of 1, 5, and 9.
The image which has been scratched in the video scratching link is the 1 st frame, the 5 th frame and the 9 th frame, and other frames can be used as the at least one image to be scratched. Or, the image to be scratched can be selected from the other frames by frame skipping, for example, the other frames are the 2 nd frame, the 3 rd frame, the 4 th frame, the 6 th frame, the 7 th frame, the 8 th frame and the 10 th frame. Then can select the 3 rd frame, the 6 th frame, the 8 th frame to carry out above-mentioned video keying as the image of waiting to matte and handle, and the 2 nd frame, the 4 th frame, the 7 th frame and the 10 th frame are handled as the image of waiting to matte when carrying out video keying again and handle, and the benefit of doing so is for the distribution of the image of being keyed on in the target video is as even as far as possible, if need show the keying effect again at this in-process, can make the demonstration of keying effect can embody the whole keying effect to the video more. For example, if the matting effect display instruction is received again after selecting the 3 rd frame, the 6 th frame, and the 8 th frame as the images to be subjected to matting and performing the above-mentioned video matting, in step S30, the 3 rd frame, the 5 th frame, the 6 th frame, the 8 th frame, and the 9 th frame may be sequentially displayed, and these frames are uniformly distributed in the target video as much as possible, so that the matting effect for the video may be represented more integrally.
In a specific embodiment, the matting at least one image to be scratched with respect to the target matting instruction to obtain the matting effect of the image to be scratched includes: determining a reference image for each image to be scratched, wherein the reference image is an image which is closest to the image time stamp to be scratched and is positioned before the time stamp of the image to be scratched and is scratched in response to the target scratching instruction in the target video; and tracking the image matting effect corresponding to the reference image to obtain the image matting effect of the image to be subjected to image matting. Taking the image of the 1 st frame, the 5 th frame and the 9 th frame as the example, and the image to be scratched is the image of the 3 rd frame, the reference image is the image of the 1 st frame, and the scratching effect of the image of the 3 rd frame can be obtained by tracking the scratching effect of the image of the 1 st frame. The disclosed embodiment improves the keying speed by tracking the keying effect.
Please refer to fig. 3, which shows a tracking schematic diagram. The upper left drawing of fig. 3 is a reference image, and the upper right drawing of fig. 3 is a mask drawing corresponding to the reference image, and the mask drawing represents the keying of the bicycle ridden person in the reference image, that is, the mask drawing embodies the keying effect. The lower left drawing of fig. 3 is an image to be scratched, and the lower right drawing of fig. 3 is a mask drawing corresponding to the image to be scratched, and the mask drawing represents a tracking result of the mask drawing of the upper right drawing of fig. 3. Obviously, the bicycle riding image to be scratched in the image to be scratched can be scratched based on the tracking result of the mask image for the upper right image. Through tracing the image matting, the image matting speed can be obviously improved.
In another embodiment, the video matting processing is performed when a new matting instruction is not received for more than a preset time and there is an image in the target video that is not subjected to matting with respect to the target matting instruction. Under the condition that no new keying instruction is received within the preset time, the computing resource can be considered to be idle, background preprocessing can be carried out in an idle state, namely, the video keying processing is carried out, and the image which is not keyed in the video is all keyed, so that the next time when the keying effect of the video is required to be displayed, the direct display can be carried out quickly, and the waiting time of a user is reduced. That is to say, when the background is idle, the frame which is not scratched can be scratched through the background preprocessing mode, so that the scratching effect of each frame in the target video can be completely displayed when the frame is played next time, and the frame skipping is not needed, so that the picture is smoother.
Under the general condition, the operation of scratching is probably with other image processing modes collaborative work, for example, after the user triggered the target to scratch the image instruction, still can further carry out other operations of transferring the picture and transferring brightness and beauty, these operations can not show and consume computing resource, can think that computing resource is comparatively idle under this condition, utilize this idle condition, can carry out the video and scratch the image and handle, the result is scratched in the buffer memory to make the broadcast of follow-up scratching effect more smooth and easy.
Of course, any of the above can store the matting effect for other purposes when the matting effect is obtained by matting the image in the target video. That is, after any image in the target video is subjected to matting aiming at the target matting instruction, the matting effect of any image is stored. The embodiment of the present disclosure does not limit the storage manner, for example, the storage may be performed based on a cache pool, or may be performed based on a page replacement algorithm, for example, an LRU algorithm is Used, and the LRU is an abbreviation of Least recent utilized. Through the mode of the image matting effect of storage, can directly show when subsequently needing to carry out the image matting effect and show, no longer need carry out image matting again, promote image matting effect display speed and efficiency.
In the embodiment of the present disclosure, when an image that is not subjected to image matting with respect to the target image matting instruction exists in the target video, a prompt message is displayed, where the prompt message is used to prompt that the target video has not completed all image matting and to display that a stuck condition exists with respect to the image matting effect of the target video. For example, when the image matting effect is displayed, the part which finishes image matting is played frame by frame, the part which does not finish image matting is played by frame skipping, and prompt information can be displayed: in the "matting process, there may be slight incarceration". The user is reminded by displaying the prompt information, and the user experience is optimized.
In the embodiment of the present disclosure, the object of image matting is not limited, and in the related art, only limited objects such as a portrait, a head portrait, and a sky in an image may be automatically identified, and only the limited objects are subjected to image matting. Different from the related art, the embodiment of the disclosure can support automatic identification of any object, so as to determine the outline of any object and perform image matting based on the outline. The functions of automatic portrait matting, head matting, background matting and sky matting in the related art are based on a single algorithm model, such as portrait matting function, which uses a trained portrait recognition model, and portrait matting based on the recognition result of the portrait recognition model. Unlike the related art, the embodiment of the present disclosure uses an algorithm model for automatically recognizing multiple objects, and the model is trained using various types of objects, so that the present disclosure has the capability of automatically recognizing various objects and extracting the edges of the various objects. The embodiment of the disclosure can randomly select objects to carry out image matting processing, is not limited to image matting aiming at certain types of objects any more, and remarkably widens the application range of image matting processing.
The embodiment of the present disclosure does not limit the triggering manner of the target image matting instruction, for example, the target image matting instruction includes a foreground main body segmentation instruction or an interactive segmentation instruction, where the foreground main body segmentation instruction is used to segment a foreground main body region in the first image; the interactive segmentation command is used for segmenting an object pointed by the interactive segmentation command in the first image. The instruction is cut apart to the prospect main part and the mutual instruction of cutting apart can be triggered through the mode that the user triggered the control that corresponds, that is to say, can realize automatic keying based on these two kinds of instructions, and through automatic keying out everything promotion keying out application scope and keying out efficiency, showing and increasing user's viscosity.
This disclosed embodiment can also provide the keying ability that becomes more meticulous, gives the user bigger keying autonomous right, promotes user's viscosity. In one embodiment, the foreground body segmentation instruction or the interactive segmentation instruction generates a first mask region, the first mask region represents a target action region of the image matting, the target image matting instruction further includes a free smearing instruction, and an action region corresponding to the free smearing instruction can be obtained; and updating the first mask area based on the action area corresponding to the free smearing instruction. In the embodiment of the present disclosure, the segmentation instruction belongs to automatic image matting based on the foreground main body, and the image matting based on the interactive segmentation instruction and the image matting based on the free smearing instruction belong to manual self-defined image matting, and the automatic image matting and the manual self-defined image matting are combined for use, so that the image matting accuracy can be significantly improved.
Taking the example of hope to scratch the head portrait in the first image, the head portrait can be roughly segmented by interactive segmentation and/or foreground body segmentation, but there may be some defects in details, for example, the hair part of the head portrait may not be segmented and the fineness is not enough. In the situation, the hair part can be smeared by triggering a free smearing instruction, an action area is determined according to a smearing track, a background segmentation model is triggered to recognize the head portrait again based on the action area and the existing first mask area to obtain a more precise head portrait recognition result, the first mask area is updated according to the head portrait recognition result, and the image matting processing is performed on the first image based on the first mask area, so that the image matting effect accurately aiming at the first image can be displayed.
Please refer to fig. 4, which illustrates a sectional interface diagram. Different image matting modes can be selected automatically and manually in the image matting interface, and the image matting modes can trigger the target image matting instruction. The "manual" mode, i.e., "brush" in fig. 4, is selected by default. The painting brush control is used for painting in the first image, and the painting brush painting area is a keying reserved area, namely a semi-transparent area of the editing state preview area. And in the editing state, selecting the painting brush, and pressing the painting brush by a finger in an upper preview area. In fig. 4, an "eraser" is further provided, and the eraser-erased area is a scratch-off area, that is, an unidentified area in the editing state preview area. In the editing state, the eraser is selected, and the marked area can be removed in the preview area. FIG. 4 is also provided with a "preview," which in the case of a control assumes a non-preview state that may display artwork for the first image and identify the reserved area, where the semi-transparent blob represents the reserved area. And when the control is opened, a preview state is presented, the preview state only displays the area after image matting, and the semitransparent color blocks on the picture are hidden. The default is the preview off state under the condition of entering the user-defined matting, and the editing and the previewing can be switched back and forth under the condition of exiting the user-defined matting. Clicking on the triangular button in fig. 4 may generate a keying effect display instruction that instructs the user to play the keying effect of the target video.
Referring to fig. 5, there is shown a keying effect display device, the device comprising:
a first display module 10 configured to perform, in a case where a keying effect display instruction for a target video is obtained, displaying a keying effect of a first image, where the first image is a first keyed image in the target video in response to the keying instruction;
a second display module 20 configured to perform a matting effect of displaying a third image in a case where the second image does not exist;
wherein the second image is an image with a time stamp behind the first image in the target video and which is scratched in response to the target scratching instruction,
the third image is an image of the target video, in which the timestamp is located after the first image and is not adjacent to the first image.
In some possible embodiments, the second display module is configured to perform:
and displaying the keying effect of a fourth image, wherein the fourth image is an image of the target video, of which the time stamp is positioned behind the third image and is not adjacent to the third image.
In some possible embodiments, the second display module is configured to perform:
and when at least one second image exists, sequentially displaying the keying effect of the second images according to the sequence of the second images in the target video.
In some possible embodiments, the second display module is configured to perform:
displaying a keying effect of a fifth image under the condition that an image which is not keyed according to the target keying instruction exists in the target video;
the fifth image is an image of the target video, whose timestamp is located after the target second image and is not adjacent to the target second image;
the target second image is the second image with the largest time stamp.
In some possible embodiments, the second display module is configured to perform:
and displaying the keying effect of a sixth image, wherein the sixth image is an image of the target video, of which the time stamp is positioned behind the fifth image and is not adjacent to the fifth image.
In some possible embodiments, the above apparatus further comprises a matting module configured to perform:
when the keying effect of the image which is already keyed is displayed, if an image which is not keyed according to the target keying instruction exists in the target video, video keying processing is performed in parallel;
the video matting processing comprises:
determining at least one image to be scratched in the target video according to the image which is scratched in response to the target scratching instruction in the target video;
and carrying out image matting aiming at the target image matting instruction on the at least one image to be subjected to image matting to obtain the image matting effect of the image to be subjected to image matting.
In some possible embodiments, the matting module is configured to perform determining a reference image for each image to be scratched, where the reference image is an image in the target video that has been scratched in response to the target scratching instruction and is closest to and before a timestamp of the image to be scratched;
and tracking the image matting effect corresponding to the reference image to obtain the image matting effect of the image to be subjected to image matting.
In some possible embodiments, the above-mentioned matting module is configured to perform
And when a new image matting instruction is not received after the preset time is exceeded and an image which is not subjected to image matting aiming at the image matting instruction exists in the target video, carrying out the image matting processing.
In some possible embodiments, the matting module is configured to perform matting for any image in the target video according to the target matting instruction, and store a matting effect of any image.
In some possible embodiments, the second display module is configured to perform:
and under the condition that an image which is not subjected to image matting aiming at the target image matting instruction exists in the target video, displaying prompt information, wherein the prompt information is used for prompting that the target video does not complete all image matting and the image matting effect of the target video displays that the image matting effect of the target video is blocked.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
FIG. 6 is a block diagram illustrating an electronic device 600 for matting according to an example embodiment.
The electronic device may be a server or a terminal device, and its internal structure diagram may be as shown in fig. 6. The electronic device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a matting effect display method.
Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and does not constitute a limitation on the electronic devices to which the disclosed aspects apply, as a particular electronic device may include more or less components than those shown, or combine certain components, or have a different arrangement of components.
In an exemplary embodiment, there is also provided an electronic device including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement a matting effect display method as in embodiments of the disclosure.
In an exemplary embodiment, a computer-readable storage medium is also provided, in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform a matting effect display method in an embodiment of the present disclosure. Specifically, the keying effect display method includes:
under the condition that a keying effect display instruction for a target video is obtained, a keying effect of a first image is displayed, wherein the first image is a first keyed image in the target video and responds to the keying instruction;
displaying the keying effect of the third image under the condition that the second image does not exist;
wherein the second image is an image with a time stamp behind the first image in the target video and which is scratched in response to the target scratching instruction,
the third image is an image of the target video, in which the timestamp is located after the first image and is not adjacent to the first image.
In some possible embodiments, after displaying the keying effect of the third image, the method further includes:
and displaying the keying effect of a fourth image, wherein the fourth image is an image of the target video, of which the time stamp is positioned behind the third image and is not adjacent to the third image.
In some possible embodiments, the method further comprises:
and when at least one second image exists, sequentially displaying the keying effect of the second images according to the sequence of the second images in the target video.
In some possible embodiments, after the sequentially displaying the keying effect of the second image, the method further includes:
displaying a keying effect of a fifth image under the condition that an image which is not keyed according to the target keying instruction exists in the target video;
the fifth image is an image of the target video, whose timestamp is located after the target second image and is not adjacent to the target second image;
the target second image is the second image with the largest time stamp.
In some possible embodiments, after the displaying the keying effect of the fifth image, the method further comprises:
and displaying the keying effect of a sixth image, wherein the sixth image is an image of the target video, of which the time stamp is positioned behind the fifth image and is not adjacent to the fifth image.
In some possible embodiments, the method further comprises:
when the keying effect of the image which is already keyed is displayed, if an image which is not keyed according to the target keying instruction exists in the target video, video keying processing is performed in parallel;
the video matting processing comprises:
determining at least one image to be scratched in the target video according to the image which is scratched in response to the target scratching instruction in the target video;
and carrying out image matting aiming at the target image matting instruction on the at least one image to be subjected to image matting to obtain the image matting effect of the image to be subjected to image matting.
In some possible embodiments, the matting the at least one image to be scratched with respect to the target scratching instruction to obtain a scratching effect of the image to be scratched, including:
determining a reference image for each image to be scratched, wherein the reference image is an image which is closest to the image time stamp to be scratched and is positioned before the time stamp of the image to be scratched and is scratched in response to the target scratching instruction in the target video;
and tracking the image matting effect corresponding to the reference image to obtain the image matting effect of the image to be subjected to image matting.
In some possible embodiments, the method further comprises:
and when a new image matting instruction is not received after the preset time is exceeded and an image which is not subjected to image matting aiming at the image matting instruction exists in the target video, carrying out the image matting processing.
In some possible embodiments, the method further comprises:
after any image in the target video is subjected to image matting aiming at the target image matting instruction, the image matting effect of any image is stored.
In some possible embodiments, the method further comprises:
and under the condition that an image which is not subjected to image matting aiming at the target image matting instruction exists in the target video, displaying prompt information, wherein the prompt information is used for prompting that the target video does not complete all image matting and the image matting effect of the target video displays that the image matting effect of the target video is blocked.
In an exemplary embodiment, there is also provided a computer program product including a computer program stored in a readable storage medium, from which at least one processor of a computer device reads and executes the computer program, so that the computer device performs the matting effect display method of the embodiments of the present disclosure.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (10)

1. A keying effect display method is characterized by comprising the following steps:
under the condition that a keying effect display instruction for a target video is obtained, displaying a keying effect of a first image, wherein the first image is a first keyed image in the target video in response to the target keying instruction;
displaying the keying effect of the third image under the condition that the second image does not exist;
wherein the second image is an image in the target video with a timestamp located after the first image and being scratched in response to the target scratching instruction,
the third image is an image of the target video, of which the timestamp is located after the first image and is not adjacent to the first image.
2. The keying effect display method of claim 1, wherein after displaying the keying effect of the third image, the method further comprises:
and displaying the keying effect of a fourth image, wherein the fourth image is an image of the target video, the timestamp of which is positioned behind the third image and is not adjacent to the third image.
3. The matting effect display method according to claim 1, characterized in that the method further comprises:
and under the condition that at least one second image exists, sequentially displaying the keying effect of the second images according to the sequence of the second images in the target video.
4. The matting effect display method according to claim 3, wherein after said sequentially displaying the matting effect of the second image, the method further comprises:
displaying a keying effect of a fifth image under the condition that an image which is not keyed to the target keying instruction exists in the target video;
the fifth image is an image, of the target video, of which the timestamp is positioned after the target second image and is not adjacent to the target second image;
the target second image is the second image with the largest time stamp.
5. The matting effect display method according to claim 4, characterized in that after said displaying the matting effect of the fifth image, the method further comprises:
and displaying the keying effect of a sixth image, wherein the sixth image is an image of the target video, the timestamp of which is positioned behind the fifth image and is not adjacent to the fifth image.
6. The matting effect display method according to any one of claims 1 to 5, characterized in that the method further comprises:
when the keying effect of the image which is already keyed is displayed, if an image which is not keyed according to the target keying instruction exists in the target video, video keying processing is performed in parallel;
the video matting processing comprises:
determining at least one image to be scratched in the target video according to the image which is scratched in response to the target scratching instruction in the target video;
and carrying out image matting aiming at the target image matting instruction on the at least one image to be subjected to image matting to obtain the image matting effect of the image to be subjected to image matting.
7. A keying effect display device, comprising:
the display device comprises a first display module and a second display module, wherein the first display module is configured to execute the step of displaying the keying effect of a first image under the condition that the keying effect display instruction for a target video is obtained, and the first image is a first keyed image in the target video in response to the target keying instruction;
a second display module configured to perform a matting effect of displaying a third image in the absence of the second image;
wherein the second image is an image in the target video with a timestamp located after the first image and being scratched in response to the target scratching instruction,
the third image is an image of the target video, of which the timestamp is located after the first image and is not adjacent to the first image.
8. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the matting effect display method according to any one of claims 1 to 6.
9. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the matting effect display method according to any one of claims 1 to 6.
10. A computer program product, characterized in that the computer program product comprises a computer program stored in a readable storage medium, from which at least one processor of a computer device reads and executes the computer program, causing the computer device to perform the matting effect display method according to any one of claims 1 to 6.
CN202210089352.3A 2022-01-25 2022-01-25 Image matting effect display method and device, storage medium and electronic equipment Pending CN114564612A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210089352.3A CN114564612A (en) 2022-01-25 2022-01-25 Image matting effect display method and device, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210089352.3A CN114564612A (en) 2022-01-25 2022-01-25 Image matting effect display method and device, storage medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN114564612A true CN114564612A (en) 2022-05-31

Family

ID=81714251

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210089352.3A Pending CN114564612A (en) 2022-01-25 2022-01-25 Image matting effect display method and device, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN114564612A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919836A (en) * 2019-03-20 2019-06-21 广州华多网络科技有限公司 Video keying processing method, video keying processing client and readable storage medium storing program for executing
CN113253890A (en) * 2021-04-02 2021-08-13 中南大学 Video image matting method, system and medium
CN114549547A (en) * 2022-01-25 2022-05-27 北京达佳互联信息技术有限公司 Image matting method and device, storage medium and electronic equipment

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919836A (en) * 2019-03-20 2019-06-21 广州华多网络科技有限公司 Video keying processing method, video keying processing client and readable storage medium storing program for executing
CN113253890A (en) * 2021-04-02 2021-08-13 中南大学 Video image matting method, system and medium
CN114549547A (en) * 2022-01-25 2022-05-27 北京达佳互联信息技术有限公司 Image matting method and device, storage medium and electronic equipment

Similar Documents

Publication Publication Date Title
WO2021232978A1 (en) Video processing method and apparatus, electronic device and computer readable medium
AU2017254848B2 (en) Image matting using deep learning
US10742900B2 (en) Method and system for providing camera effect
US11245871B1 (en) Gesture-based whiteboard
CN112752116A (en) Display method, device, terminal and storage medium of live video picture
US20210279888A1 (en) Foreground data generation method and method for applying same, related apparatus, and system
US20200265238A1 (en) Methods and Systems for Identification and Augmentation of Video Content
CN108769537A (en) Photographing method, device, terminal and readable storage medium
US20230353701A1 (en) Removing objects at image capture time
US8346006B1 (en) Real time auto-tagging system
CN113709545A (en) Video processing method and device, computer equipment and storage medium
CN113727039B (en) Video generation method and device, electronic equipment and storage medium
EP3885934A1 (en) Video search method and apparatus, computer device, and storage medium
Syahputra et al. Finger recognition as interaction media in Augmented Reality for historical buildings in Matsum and Kesawan regions of Medan City
CN114549547A (en) Image matting method and device, storage medium and electronic equipment
CN113225451A (en) Image processing method and device and electronic equipment
US11232616B2 (en) Methods and systems for performing editing operations on media
CN112367465A (en) Image output method and device and electronic equipment
CN114564612A (en) Image matting effect display method and device, storage medium and electronic equipment
CN108495038B (en) Image processing method, image processing device, storage medium and electronic equipment
CN114650370A (en) Image shooting method and device, electronic equipment and readable storage medium
CN114237800A (en) File processing method, file processing device, electronic device and medium
Zeng et al. Instant video summarization during shooting with mobile phone
CN113347355A (en) Image processing method and device and electronic equipment
CN111556251A (en) Electronic book generation method, device and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination