WO2023207522A1 - 视频合成方法、装置、设备、介质及产品 - Google Patents

视频合成方法、装置、设备、介质及产品 Download PDF

Info

Publication number
WO2023207522A1
WO2023207522A1 PCT/CN2023/085796 CN2023085796W WO2023207522A1 WO 2023207522 A1 WO2023207522 A1 WO 2023207522A1 CN 2023085796 W CN2023085796 W CN 2023085796W WO 2023207522 A1 WO2023207522 A1 WO 2023207522A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
video
target
displacement
extended
Prior art date
Application number
PCT/CN2023/085796
Other languages
English (en)
French (fr)
Inventor
杨焜傲
王婕
高晶
李秋婷
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2023207522A1 publication Critical patent/WO2023207522A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the embodiments of the present disclosure relate to the field of computer technology, and in particular, to a video synthesis method, apparatus, equipment, media and products.
  • Embodiments of the present disclosure provide a video synthesis method, device, equipment, medium and product for avatar video synthesis to overcome the technical problem of relatively single video processing methods in the prior art.
  • embodiments of the present disclosure provide a video synthesis method, including:
  • the target rendering image corresponding to the image frame is subjected to video synthesis to obtain the target video corresponding to the video to be processed.
  • an embodiment of the present disclosure provides a device for video synthesis, including:
  • An image segmentation unit used to perform image segmentation on the image frames in the video to be processed, and obtain the area image corresponding to the target object and the background image except the area image;
  • a mirror image processing unit used to determine the mirror image corresponding to the regional image
  • a first expansion unit configured to expand the region image in a first expansion direction to obtain at least one first Extended image; at least one of said first extended images includes said area image;
  • a second expansion unit configured to expand the mirror image in a second expansion direction to obtain at least one second expansion image; the first expansion direction and the second expansion direction are different;
  • An image rendering unit configured to render the background image, at least one second extended image, and at least one first extended image to obtain a target rendered image at the end of rendering;
  • a video synthesis unit is configured to perform video synthesis on the target rendering image corresponding to the image frame, and obtain the target video corresponding to the video to be processed.
  • embodiments of the present disclosure provide an electronic device, including: a processor and a memory;
  • the memory stores computer execution instructions
  • the processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the video synthesis method described in the above first aspect and various possible designs of the first aspect.
  • embodiments of the present disclosure provide a computer-readable storage medium.
  • Computer-executable instructions are stored in the computer-readable storage medium.
  • the processor executes the computer-executable instructions, the above first aspect and the first aspect are implemented. aspects of various possible designs for the described video synthesis method.
  • embodiments of the present disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the video synthesis method described in the first aspect and various possible designs of the first aspect.
  • embodiments of the present disclosure further provide a computer program that, when run by a processor, implements the video synthesis method of the first aspect and various possible designs of the first aspect.
  • Figure 1 is an application schematic diagram of a clone video synthesis method provided by an embodiment of the present disclosure
  • Figure 2 is a flow chart of an embodiment of a video synthesis method provided by an embodiment of the present disclosure
  • Figure 3 is an example diagram of a mirror image provided by an embodiment of the present disclosure.
  • Figure 4 is a flow chart of another embodiment of a video synthesis method provided by an embodiment of the present disclosure.
  • Figure 5 is an example diagram of a displacement point provided by an embodiment of the present disclosure.
  • Figure 6 is a flow chart of another embodiment of a video synthesis method provided by an embodiment of the present disclosure.
  • Figure 7 is an example diagram of a target rendering image provided by an embodiment of the present disclosure.
  • Figure 8 is a flow chart of another embodiment of a video synthesis method provided by an embodiment of the present disclosure.
  • Figure 9 is a structural example diagram of a video synthesis device provided by an embodiment of the present disclosure.
  • FIG. 10 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present disclosure.
  • a rendered image can be obtained by separately extracting, expanding and rendering the video, and using the rendered images of each video frame to synthesize the video.
  • image Expansion and rendering synthesis can be completed automatically to obtain the avatar video at the end of rendering and improve the display efficiency of the avatar video.
  • the present disclosure considers adding clones of objects in motion in the video to increase the interest of the video. Therefore, consider automatically splitting each frame of image, and then synthesizing the split video frames into the target video.
  • the area image and mirror image of the target object can be expanded in different directions, and then the expanded images can be rendered uniformly to obtain the corresponding target rendering image.
  • objects in the image can be cloned to use video synthesis, which improves the synthesis efficiency and interest of the video, and has a positive effect on increasing the playback rate of the video.
  • the image frame in the video to be processed is segmented to obtain the regional image and background image where the target object is located, the regional image is expanded in the first expansion direction and the mirror image is expanded in the second expansion direction, At least one first extended image and at least one second extended image are obtained, and through rendering of at least one first extended image, at least one second extended image and a background image, a target rendered image having a clone of the target object is obtained.
  • the target rendering image is obtained by rendering the background image, at least one second extended image, and at least one first extended image, and can display forward clones and reverse clones, thereby improving image display efficiency.
  • Perform video synthesis on the target rendering image corresponding to the image frame to obtain the target video of the video to be processed.
  • FIG. 1 is an application diagram of a video synthesis method according to the present disclosure.
  • the application schematic diagram according to the embodiment of the present disclosure may include an electronic device and a client connected to the electronic device through a local area network or a wide area network.
  • the electronic device may be a personal computer, an ordinary server, a super personal computer, a cloud server, etc. type of server, the specific type of electronic device is not too limited in this disclosure.
  • the client can be a terminal device such as a mobile phone, a tablet computer, a personal computer, a smart home appliance, a wearable device, etc. This document does not limit the specific type of the client too much.
  • the client 2 can be configured on a terminal device such as a mobile phone 21, a tablet computer 22, etc., and any client 2 can provide the cloud server 1 with a video to be processed.
  • the cloud server 1 can obtain the video to be processed sent by any client 2, and synthesize the target video based on the video synthesis method provided by the embodiment of the present disclosure.
  • cloud server 1 can send the synthesized target video to client 2.
  • the target video can be released.
  • the target video is successfully released, and other clients 3, such as the mobile phone 3 shown in Figure 1, can obtain the target video and play the target video.
  • the target video can also be recommended to other clients 3 to increase the playback rate of the target video.
  • Figure 2 is a flow chart of an embodiment of a video synthesis method provided by an embodiment of the present disclosure.
  • the method can be configured as a video synthesis device, and the video synthesis device can be located in an electronic device.
  • the video synthesis method can include the following steps:
  • image frames can be sampled from the video to be processed.
  • the sampling frequency can be set including the frame rate of the video to be processed. If the sampling frequency is set to be the same as the frame rate of the image frame, it can be ensured that each image frame in the video can be sampled, and the technical solution of the present disclosure is used for image processing to obtain the corresponding target rendering image.
  • the sampling frequency of the video to be processed can also be set according to actual needs and is not limited to being equal to the frame rate.
  • the higher the sampling frequency the greater the number of image frames collected per unit time, and the lower the possibility of discontinuous images.
  • the lower the sampling frequency the fewer the number of images collected per unit time, and the higher the possibility of discontinuous images.
  • Image frames can be sampled from the video to be processed.
  • the image frame may include at least one, and each image frame may obtain a corresponding target rendering image through steps 201-204.
  • the regional image can be an image of the area where the outline of the target object is located, or it can be an irregular image.
  • the background image may be an image corresponding to other areas in the video to be processed except for the outline of the target object.
  • the target objects may include human bodies, objects, animals and other movable objects.
  • Common target objects may include, for example, people or animals in motion.
  • a mirror image can be obtained by mirror flipping the area image.
  • the image content of the mirror image is the same as the area image, but the display direction is opposite.
  • the mirror image may include any one of a horizontal mirror image, a vertical mirror image, and a diagonal mirror image.
  • the water surface mirror image refers to mirroring the left and right parts of the image with the vertical central axis of the image as the center.
  • Vertically mirroring an image can be done by mirroring the upper and lower parts of the image centered on the horizontal central axis of the image.
  • the diagonal mirror image can be a mirror swap centered on the intersection of the horizontal central axis and the vertical central axis of the image, which is equivalent to performing horizontal mirroring and vertical mirroring of the image successively.
  • the vertex A of which is on the left side of the side length B.
  • the vertex A' is on the right side of the side length B'.
  • first expansion direction and the second expansion direction are different.
  • the first expansion direction and the second expansion direction may be opposite. That is, the angle between the first expansion direction and the second expansion direction is 180 degrees.
  • the first expansion direction and the second expansion direction can also be determined based on the two included angles formed by the vertical line of the horizontal line.
  • the vertical line of the horizontal line can be used as the common side of the two included angles, and the other side of the two included angles points to The directions may be the first expansion direction and the second expansion direction respectively.
  • two included angles are represented by included angle A and included angle B.
  • the angles between angle A and angle B may be equal or unequal, and the angle between angle A and angle B is between 0 and 180 degrees.
  • the included angle is 90 degrees
  • the first expansion direction and the second expansion direction are opposite.
  • Angle A and When the included angle B is the same, the first expansion direction and the second expansion direction are symmetrical with respect to the vertical line of the horizontal line.
  • the first expansion direction may include the vertical right side of the horizontal line
  • the second expansion direction may include the vertical right side of the horizontal line
  • the horizontal line may include a horizontal line corresponding to the center point of the target object.
  • the left side of the image on the display screen that can be viewed by human eyes can be reversed, and the left side of the image on the display screen that can be viewed by human eyes can be reversed.
  • the right direction can be positive. Forward and reverse may specifically refer to the left or right expansion of the image.
  • the area image can be expanded in the first expansion direction to the right to obtain at least one first expanded image, and can be expanded in the second expansion direction to the left to obtain at least one second expanded image.
  • the target rendering image may be obtained by rendering the background image, at least one second extended image, and at least one first extended image respectively.
  • it may include a background image, at least one second extended image, and at least one first extended image.
  • the rendered image obtained at the end of layer rendering.
  • the target video can be obtained by synthesizing target rendering images corresponding to multiple image frames.
  • the target rendering image corresponding to the image frame can be video synthesized according to the timestamp of the image frame.
  • the timestamp of an image frame is the timestamp of its corresponding target rendered image.
  • the frame rate of the target video can be determined based on the frame rate of the video to be processed, and the synthesis time of the video can be accurately controlled through the frame rate of the video.
  • the image frame in the video to be processed is segmented to obtain the area image and background image where the target object is located, and the area image and the mirror image of the area image are used for image expansion to obtain at least one first extended image and at least one
  • the second extended image obtains a target rendering image having a clone of the target object through the rendering of at least one first extended image, at least one second extended image and a background image.
  • the target rendering image is obtained by rendering the background image, at least one second extended image, and at least one first extended image, and can display forward clones and reverse clones, thereby improving image display efficiency.
  • the target rendering image corresponding to the image frame can be video synthesized to obtain the target video of the video to be processed.
  • the video to be processed involved in the present disclosure may be collected in real time, by extracting image frames from the video collected in real time. Then, the image frame is segmented into regions of the target object, and the region image obtained by segmentation is used to perform mirror flipping to realize the expansion of the forward region image and the expansion of the mirror image, and obtain at least one first expanded image and at least one second Extended image.
  • a target rendering image is obtained by rendering a background image other than the area image, at least a first extended image and at least one second extended image, so as to use the target rendered image to perform video synthesis. This process is completed automatically and does not require manual intervention, such as manual copying of common drawing software.
  • the synthesis efficiency of the video is high, which can solve the problem of low synthesis efficiency of manually completing the avatar video in the existing technology. question.
  • the image frames in the video to be processed are segmented to obtain the regional image and background image where the target object is located, and the regional image and the mirror image of the regional image are used for image expansion.
  • the regional image can be expanded through the first expansion direction. Expand, obtain at least one first expanded image, expand the mirror image through the second expansion direction, obtain at least one second expanded image.
  • the target object can be cloned.
  • the rendering of at least one first extended image, at least one second extended image and a background image a target object having a clone of the target object is obtained. Marker rendered image.
  • the target rendering image is obtained by rendering the background image, at least one second extended image and at least one first extended image, and can display extended images in the first extension direction and the second extension direction, thereby improving the display efficiency and interest of the image.
  • By automatically completing image rendering and automatic synthesis of the target video the efficiency and accuracy of video acquisition can be improved, and the playback speed and efficiency of the target video can be ensured. Effective promotion.
  • a flow chart of yet another embodiment of a video synthesis method provided by an embodiment of the present disclosure is different from other embodiments in that, in the Expanding the regional image in an expansion direction to obtain at least one first expanded image may include:
  • Displacement speed can refer to the moving speed of the target object in the video to be processed. For example, assuming that the target object is at the N position when playing the video, and moves to the M position when the second frame of the image is played, the displacement speed of the target object can be combined with the time difference of the two image frames based on the difference between the N position and the M position. Calculated.
  • the regional image can refer to marking the pixels belonging to the target object in the image frame to obtain the pixels of the regional image in the image frame.
  • the coordinates of the pixels corresponding to the regional image can also be represented by the pixel coordinate system of the image frame. Pixels that are not marked as area images are pixels of the background image.
  • the original position point of the regional image may refer to the pixel coordinate point corresponding to the key point of the target object in the image frame.
  • Key points can refer to pixel points that can confirm the position of the regional image.
  • the coordinates of the pixel points are represented by the pixel coordinate system of the entire image frame.
  • the original position point may include, for example, the coordinate point of the center point of the area image in the image frame.
  • the target object is a human body
  • it can also be a key position point in the human body, such as the coordinates of a pixel point such as the center of the eyebrow point, the tip of the nose point, the center point, etc.
  • the position meaning corresponding to the original position point is not limited here.
  • the forward displacement point may include a pixel coordinate point located in the image frame obtained after the original position point is displaced. Both the original position point and the forward displacement point may include coordinate points located in the pixel coordinate system of the image frame.
  • the first expansion direction may include a forward displacement angle.
  • the first expansion direction may be determined by the trajectory of the target object in the video frame. For example, taking the same example of moving from the N position to the M position, the line connecting the two positions may have an angle relative to the horizontal line or the vertical line, and the angle is the positive displacement angle. Assume that the forward displacement angle is 45 degrees.
  • a straight line can be moved to the upper right and has an angle of 45 degrees with the horizontal line.
  • the moving distance is calculated according to the displacement speed to obtain a point, which is a forward displacement point.
  • P0 is the original position point
  • P0 is displaced 45 degrees in the forward direction
  • the obtained forward displacement point can be P1.
  • Copying the area image to the forward displacement point may include moving the pixel value corresponding to the pixel point of the area image to the corresponding coordinate point according to the forward displacement point. That is, it may be based on the coordinate point of the area image in the image frame. and the forward displacement point to determine the forward coordinate point of the first extended image in the image frame. By assigning the pixel value at the coordinate point in the regional image to the corresponding forward coordinate point, the forward coordinate point corresponding to The first extended image composed of pixel values.
  • the regional image may include multiple pixels, and each pixel may be moved one by one according to the forward displacement point to obtain a first extended image composed of multiple moved pixels.
  • the coordinate value of other pixel points is set to the transparent attribute.
  • they are all transparent and do not exist. In terms of color, it does not cause display occlusion of other images, such as background images.
  • the forward displacement point may include the original position point, that is, without displacing the regional image, generating a first extended image corresponding to the original position point can ensure that the regional image at the original position can be displayed normally.
  • the coordinates of the pixel points of the first extended image corresponding to the original position point do not change.
  • At least one accurate forward displacement point can be obtained by obtaining the displacement velocity of the target object and performing forward displacement based on the original position point of the target object.
  • the regional image is confirmed according to the forward displacement point, so as to obtain the first extended image corresponding to the forward displacement point, improve the positioning accuracy of the first extended image, and obtain a higher positioning accuracy.
  • At least one forward displacement point corresponds to the first extended image respectively.
  • displacement processing is performed according to the displacement speed and the first expansion direction to obtain at least one forward displacement point containing the original position point, which may include:
  • the forward displacement amount and the forward offset step of the area image are calculated in the first expansion direction.
  • At least one forward position point is determined according to each forward offset step and the number of forward displacements, and at least one forward displacement point composed of the original position point and at least one forward position point is obtained.
  • the area width of the area image may refer to the horizontal width of the object outline area.
  • Image width can also refer to the horizontal width of the image frame.
  • the amount of forward displacement can be automatically calculated based on the area width and image width. For example, the quotient of half the image width and the area width can be calculated, and then the obtained result is rounded to obtain the amount of forward displacement.
  • the forward displacement amount can also be preset. For example, it can be set by operation and maintenance personnel or set and obtained by front-end users who use the video synthesis method. In more common scenarios, the number of forward displacements can be set to 3.
  • the forward offset step size can be determined based on the quotient of half the image width, the width difference of the area width and the amount of forward displacement, with the displacement speed as a constraint. Usually the quotient of the width difference and the amount of forward displacement can be calculated, and then the initial forward offset step can be obtained by rounding the quotient result. If the displacement speed is greater than a certain threshold, the step size is reduced. If the displacement speed is less than A certain threshold, the step size increases.
  • the amount of forward displacement and the forward displacement step in the first expansion direction can be accurately calculated according to the area width of the area image and the image width of the image frame.
  • the forward displacement can be quantity and forward offset step, accurately position the forward position point, and obtain a forward displacement point with higher position accuracy.
  • a flow chart of yet another embodiment of a video synthesis method provided by an embodiment of the present disclosure is different from other embodiments in that, in the Expand the mirror image in the second expansion direction to obtain at least one second expanded image, including:
  • the regional image can refer to marking the pixels belonging to the target object in the image frame to obtain the pixels of the regional image in the image frame.
  • the coordinates of the pixels corresponding to the regional image can also be represented by the pixel coordinate system of the image frame. Pixels that are not marked as area images are pixels of the background image.
  • the original position point of the regional image may refer to the pixel coordinate point corresponding to the key point of the target object in the image frame.
  • Key points can refer to pixel points that can confirm the position of the regional image.
  • the coordinates of the pixel points are represented by the pixel coordinate system of the entire image frame.
  • the original position point may include, for example, the coordinate point of the center point of the area image in the image frame.
  • the target object is a human body
  • it can also be a key position point in the human body, such as the coordinates of a pixel point such as the center of the eyebrow point, the tip of the nose point, the center point, etc.
  • the meaning of the position corresponding to the original position point is not limited here.
  • the reverse displacement point may include a pixel coordinate point located in the image frame obtained after the original position point is displaced. Both the original position point and the reverse displacement point may include coordinate points located in the pixel coordinate system of the image frame.
  • the second expansion direction may include a reverse displacement angle.
  • the second extension direction may be determined by the trajectory of the target object in the video frame. For example, taking the same example of moving from the N position to the M position, the line connecting the two positions may have an included angle relative to the horizontal line or vertical line, and the included angle is the reverse displacement angle.
  • the reverse displacement process is similar to the forward displacement and will not be described again here.
  • 603 Copy the mirror image to the reverse displacement point, and obtain the second extended image corresponding to the mirror image at at least one reverse displacement point.
  • Copying the area image to the reverse displacement point may include moving the pixel value corresponding to the pixel point of the area image to the corresponding coordinate point according to the reverse displacement point, that is, it may be based on the coordinate point of the area image in the image frame. and the reverse displacement point to determine the reverse coordinate point of the second extended image in the image frame.
  • the reverse coordinate point can be obtained.
  • the second extended image composed of corresponding pixel values.
  • the regional image may include multiple pixels, and each pixel may be moved one by one according to the reverse displacement point to obtain a second extended image composed of multiple moved pixels.
  • the coordinates of other pixels are set to the transparent attribute.
  • they are all transparent and have no color, so as not to be inconsistent with other images, such as The background image creates display occlusion.
  • At least one accurate reverse displacement point can be obtained by obtaining the displacement velocity of the target object and performing reverse displacement based on the original position point of the target object.
  • the regional image is confirmed according to the reverse displacement point, so as to obtain the second extended image corresponding to the reverse displacement point, improve the positioning accuracy of the second extended image, and obtain a higher positioning accuracy.
  • At least one reverse displacement point corresponds to a second extended image respectively.
  • the original position point of the regional image is used as the basis, displacement processing is performed based on the displacement velocity and the second expansion direction, and at least one reverse displacement point corresponding to the original position point is obtained, including:
  • At least one reverse position point is determined according to the reverse offset step size and the reverse offset number, and at least one reverse displacement point composed of at least one reverse position point is obtained.
  • the area width of the area image may refer to the horizontal width of the object outline area.
  • Image width can also refer to the horizontal width of the image frame.
  • the amount of reverse displacement can be automatically calculated based on the area width and image width. For example, the quotient of half the image width and the area width can be calculated, and then the obtained result is rounded to obtain the amount of reverse displacement.
  • the reverse displacement amount can also be preset. For example, it can be set by operation and maintenance personnel or set and obtained by front-end users who use the video synthesis method. In more common scenarios, the number of reverse displacements can be set to 3.
  • the reverse offset step size can be determined based on the quotient of half the image width, the width difference of the area width and the reverse displacement amount, with the displacement speed as a constraint. Usually the quotient of the width difference and the amount of reverse displacement can be calculated, and then the quotient The initial reverse offset step size can be obtained by rounding the result. If the displacement speed is greater than a certain threshold, the step size is reduced. If the displacement speed is less than a certain threshold, the step size is increased.
  • the amount of reverse displacement and the reverse offset step in the second expansion direction can be accurately calculated according to the area width of the area image and the image width of the image frame.
  • the displacement amount and the reverse offset step can accurately locate the reverse position point and obtain the reverse displacement point with higher position accuracy.
  • determining the corresponding displacement velocity of the target object in the image frame includes:
  • calculating the displacement amount corresponding to the target object according to the first position and the second position may include: determining the frame rate of the video to be processed, calculating the time interval between the two image frames according to the frame rate of the video to be processed, and Calculate the position difference between the first position and the second position.
  • the position difference can refer to the absolute difference between the first position and the second position. Specifically, it can be obtained by triangular calculation of the lateral displacement difference and the longitudinal displacement difference of the two positions, that is, the lateral difference and the longitudinal difference are calculated respectively.
  • the coordinates are the two right-angled sides of a right-angled triangle, and the displacement is the length of the hypotenuse of the right-angled triangle.
  • the quotient of displacement and time interval is the displacement speed.
  • the first position of the target object in the previous image frame and the second position of the current image frame can be obtained, and the corresponding displacement of the target object can be calculated by calculating the first position and the second position.
  • the displacement speed of the target image can be accurately calculated.
  • the displacement velocity of the target object can be accurately calculated through the displacement amount and time interval between two adjacent image frames, and high-precision displacement velocity can be obtained.
  • the background image, at least one second extended image and at least one first extended image are rendered to obtain the Target rendered image, which can include:
  • the background image being in the first rendering order
  • at least one second extended image being in the second rendering order
  • at least one first extended image being in the third rendering order
  • the background image, the at least one second extended image and the at least one first rendering are sequentially superimposed and rendered. Extend the image to obtain the target rendered image at the end of rendering.
  • the background image, at least one second extended image, and at least one first extended image are rendered, and obtaining the target rendered image at the end of rendering may include: rendering the background image, at least one second extended image, and at least one first extended image.
  • the extended images are rendered separately to obtain the target rendered image at the end of rendering.
  • the target rendering image may be obtained by rendering the background image, at least one second extended image, and at least one first extended image respectively.
  • the rendering order can be set separately for the background image, at least one second extended image, and at least one first extended image to achieve accurate rendering of the target rendering image and avoid image display overlay due to random rendering between images. question to ensure the validity of the display.
  • sequentially overlaying and rendering the background image, at least one second extended image, and at least one first extended image to obtain the target rendering image at the end of rendering may include:
  • At least one first extended image is rendered sequentially until the rendering of the last first extended image is completed, and the target rendering image is obtained.
  • the reverse center distance between the second extended image and the center point of the background image can be calculated and obtained by calculating the reverse displacement point of the second extended image and the center point of the background image.
  • the forward center distance between the center point of the first extended image and the background image can be calculated and obtained by calculating the forward displacement point of the first extended image and the center point of the background image.
  • the target rendered image may be obtained by multi-layer rendering of the background image, at least one second extended image, and at least one first extended image.
  • the background image may be rendered on the lowest layer
  • the second extended image may be rendered on the upper layer of the background image
  • the first extended image may be rendered on the upper layer of the second extended image.
  • There is also a rendering order in the rendering process of at least one second extended image that is, the second extended image with the largest distance from the center point of the image is rendered first. As the distance decreases, the rendering order goes later.
  • the rendering process of at least one first extended image is similar, that is, the first extended image with the largest distance from the center point of the image is rendered first, and as the distance decreases, the rendering order becomes later.
  • At least one second extended image and at least one first extended image are rendered in the same batch, and are rendered directly according to the distance from the center point, starting from the image corresponding to the largest distance until the rendering of the last first extended image is completed.
  • the first extended image includes a region image whose distance from the center point is the smallest.
  • the area image is shown as 701
  • the area image is shown as 701
  • One extended image is 701-703, respectively
  • at least one second extended image is 704-705, respectively.
  • the background image is rendered at the lowest layer to ensure that the background image is displayed normally.
  • the first extended image and the second extended image may be rendered on an upper layer of the background image.
  • the second extended image may be rendered sequentially in descending order of reverse center distance and forward center distance.
  • the first extended image is rendered to the upper layer, so that the rendering level of the image is related to the distance, achieving accurate rendering of the image, ensuring that the regional image can be rendered on the uppermost layer, and obtaining an accurately rendered target rendering image.
  • a texture image can be set for a regional image.
  • the video synthesis method in the present disclosure may further include:
  • the sticker image can be specified by the user who provides the video to be processed, and can refer to various types of stickers such as cartoon images, landscape images, and photo stickers.
  • the location of the map can also be specified by the user.
  • the map position point can be positioned based on the original position point.
  • the position point of the head vertex in the regional image can be determined based on the original position point of the regional image in the image frame, and the position point of the head vertex is used as the map position.
  • the texture position of the texture image in the image frame can be determined based on the original position point of the area image in the image frame.
  • the texture image can be rendered into the target rendering image according to the texture position, the target texture image can be obtained, and the target texture corresponding to the image frame can be By performing video synthesis on the image, the texture target video corresponding to the video to be processed can be obtained.
  • the content of the target video can be made more comprehensive and the video display efficiency can be improved.
  • perform image segmentation on the image frames in the video to be processed to obtain the regional image corresponding to the target object and the background image except the regional image including:
  • the object contour area is used to segment the image frame, and the area image corresponding to the object outline area and the background image except the area image are obtained.
  • a contour extraction algorithm can be used to identify the object contour area of the target object in the image frame.
  • the contour extraction algorithm may be a common contour extraction algorithm, and the specific type of the contour extraction algorithm is not too limited in this disclosure.
  • image segmentation can refer to setting the pixels surrounded by the object outline area as object identifiers, and setting other pixels outside the object outline area as background identifiers. The pixels of the area image and the background image can be distinguished through the identification of pixels.
  • image frames can be extracted from the video to be processed, and the object outline area of the target object in the image frame can be identified.
  • the object outline area can be used to segment the image frame to obtain an accurate regional image of the object outline area. and background image.
  • Image segmentation can achieve accurate segmentation of images and accurate acquisition of regional images and background images.
  • the approach could also include:
  • the video to be processed is obtained.
  • the video to be processed can be collected in real time or read from stored video.
  • Outputting the target video for the first user may include using an output device to output the target video for the first user or sending the target video to a user equipment corresponding to the first user, and the output device in the user equipment outputs the target video.
  • the video to be processed provided by the first user can be obtained, and after the target video is obtained, the target video can also be output for the first user.
  • the acquisition of the video to be processed and the output of the target video can be achieved, so as to provide the first user with the target video in a more efficient and automated mode.
  • the target video for the first user after outputting the target video for the first user, it also includes:
  • the target video is published to the user video account of the first user, and the target video is read and played by the second user device of the second user.
  • the electronic device may provide the first user with publishing controls of the target video.
  • the user equipment of the first user detects that the first user triggers the publishing control
  • the user equipment can send a publishing request to the electronic device.
  • the electronic device may respond to the publishing request to store the target video in the user video account of the first user.
  • the electronic device may store the target video and establish a video name or video identification of the target video and the account name of the first user. Establish a storage association between them.
  • a user video account can correspond to multiple videos, which can include target videos.
  • the second user device needs to play the target video, it determines the identity video from the user video account of the first user and sends the target video to the second user device. Among them, the target video is played once, and the number of times the target video is played increases accordingly.
  • the publishing request triggered by the first user for the target video can be detected, the target video can be published to the user video account of the first user, and the target video can be read and played by the second user device of the second user. Pass first The user's control over the release of the target video can achieve effective release of the target video.
  • detecting the target video collection request triggered by the first user, and obtaining the video to be processed includes:
  • Output the target video for the first user including:
  • the electronic device can directly interact with the first user, collect the video to be processed, and synthesize the target video corresponding to the video to be processed based on the video synthesis method of the present disclosure, so as to complete the video processing in a single electronic device. collection and processing to improve the utilization rate of electronic equipment.
  • the video to be processed sent by the first user equipment can be received, and the video to be processed can be collected by the user equipment of the first user.
  • the generated target video can be sent to the first user device, thereby realizing feedback of the target video to the first user device. Realize the separation of target video generation and output, reduce the processing pressure of the first user equipment, and ensure the normal operation of the first user equipment.
  • using electronic equipment with stronger computing power to complete the generation of the target video can make the generation efficiency and stability of the target video higher, ensuring the generation efficiency and success rate of the target video.
  • the video synthesis device 900 may include:
  • Image segmentation unit 901 used to perform image segmentation on the image frames in the video to be processed, and obtain the regional image corresponding to the target object and the background image except the regional image;
  • Mirror processing unit 902 used to determine the mirror image corresponding to the regional image
  • First expansion unit 903 configured to expand the regional image in the first expansion direction to obtain at least one first expanded image; at least one first expanded image includes a regional image;
  • the second expansion unit 904 is configured to expand the mirror image in the second expansion direction to obtain at least one second expansion image; the first expansion direction and the second expansion direction are different;
  • Image rendering unit 905 used to render the background image, at least one second extended image, and at least one first extended image to obtain the target rendering image at the end of rendering;
  • Video synthesis unit 906 used to perform video synthesis on the target rendering image corresponding to the image frame, and obtain the target video corresponding to the video to be processed.
  • the first expansion unit may include:
  • the speed determination module is used to determine the displacement speed of the target object corresponding to the image frame
  • the first expansion module is configured to use the original position point of the regional image as a basis, perform displacement processing according to the displacement speed and the first expansion direction, and obtain at least one forward displacement point including the original position point;
  • the first determination module is used to copy the regional image to the forward displacement point and obtain the first extended image respectively corresponding to the regional image at at least one forward displacement point.
  • the first extension module may include:
  • the first determination sub-module is used to determine the area width of the area image and the image width of the image frame;
  • the first calculation submodule is used to calculate the forward displacement amount and the forward offset step of the area image in the first expansion direction based on the area width and image width, combined with the displacement speed;
  • the forward displacement submodule is used to calculate the forward offset step size and forward displacement amount based on the original position of the regional image. Determine at least one forward position point, and obtain at least one forward displacement point composed of the original position point and at least one forward position point.
  • the second expansion unit may include:
  • the speed determination module is used to determine the displacement speed of the target object corresponding to the image frame
  • the second expansion module is used to use the original position point of the regional image as the basis, perform displacement processing based on the displacement velocity and the second expansion direction, and obtain at least one reverse displacement point corresponding to the original position point;
  • the second determination module is used to copy the mirror image to the reverse displacement point and obtain the second extended image corresponding to the mirror image at at least one reverse displacement point.
  • the second expansion module includes:
  • the second determination sub-module is used to determine the area width of the area image and the image width of the image frame;
  • the second calculation submodule is used to calculate the reverse displacement amount and the reverse offset step of the area image in the second expansion direction based on the area width and image width, combined with the displacement speed;
  • the reverse displacement submodule is used to determine at least one reverse position point according to the original position of the regional image, according to the reverse offset step size and the reverse offset number, and obtain at least one reverse position point composed of at least one reverse position point. Displacement point.
  • the speed determination module includes:
  • the third determination module is used to determine the previous image frame corresponding to the image frame in the video to be processed
  • the fourth determination module is used to determine the first position of the target object in the previous image frame and the second position in the image frame;
  • a displacement calculation module used to calculate the corresponding displacement of the target object based on the first position and the second position
  • the speed calculation module is used to calculate the corresponding displacement speed of the target object in the image frame by using the image frame display time interval and displacement amount of the video to be processed.
  • the image rendering unit may include:
  • a sequential rendering module configured to superimpose and render the background image and at least one second extension in sequence according to the background image as the first rendering order, the at least one second extension image as the second rendering order, and the at least one first extension image as the third rendering order. image and at least one first extended image to obtain the target rendered image at the end of rendering.
  • the sequential rendering module includes:
  • the first rendering module is used to render the background image at the bottom layer
  • a first distance module configured to determine the reverse center distance corresponding to the center point of the at least one second extended image and the background image
  • a second rendering module configured to sequentially render at least one second extended image in descending order of reverse center distance
  • a second distance module configured to determine the forward center distance corresponding to the center point of at least one first extended image and the background image
  • the second rendering module is configured to sequentially render at least one first extended image in order from large to small forward center distance until the rendering of the last first extended image is completed to obtain the target rendering image.
  • the device further includes:
  • the texture acquisition unit is used to acquire the texture image selected by the user for the target object
  • a position determination unit used to determine the mapping position of the texture image in the image frame based on the original position point of the area image in the image frame;
  • Image mapping unit used to render the texture image into the target rendering image according to the texture position to obtain the target texture image picture
  • the texture video unit is used to perform video synthesis on the target texture image corresponding to the image frame, and obtain the texture target video corresponding to the video to be processed.
  • the image segmentation unit includes:
  • Image extraction module used to extract image frames from the video to be processed
  • the area recognition module is used to identify the object outline area of the target object in the image frame
  • the image segmentation module is used to segment the image frame using the object outline area, and obtain the area image corresponding to the object outline area and the background image except the area image.
  • the device further includes:
  • a video acquisition unit configured to acquire the video to be processed based on the target video acquisition request triggered by the first user
  • a video output unit is used to output the target video for the first user.
  • the device may further include:
  • a publishing detection unit configured to detect a publishing request triggered by the first user for the target video
  • the request response unit is configured to publish the target video to the user video account of the first user in response to the publishing request, and the target video can be read and played by the second user device of the second user.
  • the video acquisition unit includes:
  • the video receiving module is used to receive the video to be processed sent by the first user equipment; the video to be processed is collected by the user equipment of the first user when the user triggers the target video collection request;
  • Video output unit including:
  • the video sending module is used to send the target video to the first user equipment, and control the first user equipment to play the target video.
  • the device provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principles and technical effects are similar, and will not be described again in this embodiment.
  • embodiments of the present disclosure also provide an electronic device.
  • Reference 10 shows a schematic structural diagram of an electronic device 1000 suitable for implementing embodiments of the present disclosure.
  • the electronic device 1000 may be a terminal device or a server.
  • terminal devices may include but are not limited to mobile phones, laptops, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA for short), tablet computers (Portable Android Device, PAD for short), portable multimedia players (Portable Mobile terminals such as Media Player (PMP for short), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc.
  • PDA Personal Digital Assistant
  • PDA Personal Digital Assistant
  • PAD Personal Android Device
  • portable multimedia players Portable Mobile terminals such as Media Player (PMP for short
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG. 10 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
  • the electronic device 1000 may include a processing device (such as a central processing unit, a graphics processor, etc.) 1001, which may process data according to a program stored in a read-only memory (Read Only Memory, ROM for short) 1002 or from a storage device. 1008 loads the program in the random access memory (Random Access Memory, RAM for short) 1003 to perform various appropriate actions and processes. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 are also stored.
  • the processing device 1001, ROM 1002 and RAM 1003 are connected to each other via a bus 1004.
  • An input/output (I/O) interface 1005 is also connected to bus 1004.
  • the following devices can be connected to the I/O interface 1005: input devices 1006 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a Liquid Crystal Display (LCD) ), an output device 1007 such as a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. Communication device 1009 may allow electronic device 1000 to communicate wirelessly with other devices. or wired communication to exchange data.
  • FIG. 10 illustrates electronic device 1000 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via the communication device 1009, or from the storage device 1008, or from the ROM 1002.
  • the processing device 1001 When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.
  • Embodiments of the present disclosure also include a computer program that performs the above functions defined in the methods of the embodiments of the present disclosure when run by a processor.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmd read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device When the one or more programs are executed by the electronic device, the electronic device performs the method shown in the above embodiment.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional Procedural programming language—such as "C" or a similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or it can be connected to an external computer Computer (e.g. connected via the Internet using an Internet service provider).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each box in the flowchart or block diagram may represent a A module, program segment, or part of code that contains one or more executable instructions for implementing a specified logical function.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure can be implemented in software or hardware.
  • the name of the unit does not constitute a limitation on the unit itself under certain circumstances.
  • the first acquisition unit can also be described as "the unit that acquires at least two Internet Protocol addresses.”
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs Systems on Chips
  • CPLD Complex Programmable Logical device
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM portable compact disk read-only memory
  • magnetic storage device or any suitable combination of the above.
  • a target video synthesis method including:
  • the mirror image is expanded in a second expansion direction to obtain at least one second expanded image; the first expansion direction and the second expansion direction are different.
  • performing expansion processing on the area image in the first expansion direction to obtain at least one first expanded image includes:
  • displacement processing is performed according to the displacement speed and the first expansion direction, and obtaining at least one forward displacement point including the original position point includes:
  • At least one forward position point is determined according to the forward offset step size and the forward displacement amount, and at least one forward displacement point composed of the original position point and at least one forward position point is obtained.
  • performing expansion processing on the mirror image in the second expansion direction to obtain at least one second expanded image includes:
  • taking the original position point of the area image as a basis, performing displacement processing based on the displacement velocity and the second expansion direction, and obtaining at least one reverse displacement point corresponding to the original position point including:
  • At least one reverse position point is determined according to the reverse offset step size and the reverse offset number, and at least one reverse displacement point composed of at least one reverse position point is obtained.
  • determining the displacement velocity of the target object corresponding to the image frame includes:
  • rendering the background image, at least one second extended image, and at least one first extended image to obtain a target rendered image at the end of rendering includes:
  • the background image being in the first rendering order
  • at least one second extended image being in the second rendering order
  • at least one first extended image being in the third rendering order
  • the background image, the at least one second extended image and the at least one first rendering are sequentially superimposed and rendered. Extend the image to obtain the target rendered image at the end of rendering.
  • the rendering background image, at least one second extended image, and at least one first extended image are sequentially superimposed to obtain a target rendered image at the end of rendering, including:
  • At least one first extended image is rendered in sequence until the last Once the extended image rendering is completed, the target rendering image is obtained.
  • the method further includes:
  • the target texture image corresponding to the image frame is video synthesized to obtain the texture target video corresponding to the video to be processed.
  • image segmentation is performed on the image frame in the video to be processed, and the regional image corresponding to the target object and the background image except the regional image are obtained, including:
  • the object contour area is used to segment the image frame, and the area image corresponding to the object outline area and the background image except the area image are obtained.
  • it further includes:
  • the method after outputting the target video for the first user, the method further includes:
  • the target video is published to the user video account of the first user, and the target video is read and played by the second user device of the second user.
  • obtaining the video to be processed includes:
  • Output the target video for the first user including:
  • a device for video synthesis including:
  • the image segmentation unit is used to perform image segmentation on the image frames in the video to be processed, and obtain the regional image corresponding to the target object and the background image except the regional image;
  • Mirror processing unit used to determine the mirror image corresponding to the regional image
  • a first expansion unit configured to expand the regional image in the first expansion direction to obtain at least one first expanded image; the at least one first expanded image includes a regional image;
  • the second expansion unit is configured to expand the mirror image in the second expansion direction to obtain at least one second expansion image; the first expansion direction and the second expansion direction are different.
  • An image rendering unit configured to render the background image, at least one second extended image, and at least one first extended image to obtain a target rendering image at the end of rendering;
  • the video synthesis unit is used to perform video synthesis on the target rendering image corresponding to the image frame to obtain the corresponding video to be processed. target video.
  • an electronic device including: at least one processor and a memory;
  • Memory stores instructions for execution by the computer
  • At least one processor executes computer execution instructions stored in the memory, so that at least one processor executes the above first aspect and various possible designed video synthesis methods of the first aspect.
  • a computer-readable storage medium is provided.
  • Computer-executable instructions are stored in the computer-readable storage medium.
  • the processor executes the computer-executed instructions, the first step above is implemented. aspects as well as various possible designs of video synthesis methods in the first aspect.
  • a computer program product including a computer program.
  • the computer program When the computer program is executed by a processor, the video of the above first aspect and various possible designs of the first aspect are implemented. resolve resolution.
  • embodiments of the present disclosure also provide a computer program.
  • the computer program is run by a processor, the video synthesis method of the first aspect and various possible designs of the first aspect are implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)
  • Image Generation (AREA)

Abstract

本公开实施例提供一种视频合成方法、装置、设备、介质及产品。该方法包括：对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除区域图像之外的背景图像；确定区域图像对应的镜像图像；在第一扩展方向对所述区域图像进行扩展处理，获得至少一个第一扩展图像；至少一个第一扩展图像包括区域图像；在第二扩展方向对所述镜像图像进行扩展处理，获得至少一个第二扩展图像；对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像；将图像帧对应的目标渲染图像进行视频合成，获得待处理视频对应的目标视频。通过对视频图像的检测和扩展解决了视频处理方式较为单一的问题。

Description

视频合成方法、装置、设备、介质及产品
相关申请交叉引用
本申请要求于2022年04月29日提交中国专利局、申请号为202210476242.2、发明名称为“视频合成方法、装置、设备、介质及产品”的中国专利申请的优先权，其全部内容通过引用并入本文。
技术领域
本公开实施例涉及计算机技术领域，尤其涉及一种视频合成方法、装置、设备、介质及产品。
背景技术
随着视频领域的迅速发展，短视频和直播视频等视频播放领域中对增加视频的趣味性需求日益增加。特别是在视频的实时采集场景中，目前的视频处理方式多是在视频中增加弹幕、贴图等趣味性内容。
但是，现有的视频趣味性提升的手段，对视频的处理方式较为单一，内容不够丰富，导致视频的传播速度不高，利用率不高。
发明内容
本公开实施例提供一种分身视频合成的视频合成方法、装置、设备、介质及产品，以克服现有技术中视频处理方式较为单一的技术问题。
第一方面，本公开实施例提供一种视频合成方法，包括：
对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除所述区域图像之外的背景图像；
确定所述区域图像对应的镜像图像；
在第一扩展方向对所述区域图像进行扩展处理，获得至少一个第一扩展图像；至少一个所述第一扩展图像包括所述区域图像；
在第二扩展方向对所述镜像图像进行扩展处理，获得至少一个第二扩展图像；所述第一扩展方向和所述第二扩展方向不同；
对所述背景图像、至少一个所述第二扩展图像和至少一个所述第一扩展图像进行渲染，获得渲染结束时的目标渲染图像；
将所述图像帧对应的目标渲染图像进行视频合成，获得所述待处理视频对应的目标视频。
第二方面，本公开实施例提供一种用于视频合成装置，包括：
图像分割单元，用于对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除所述区域图像之外的背景图像；
镜像处理单元，用于确定所述区域图像对应的镜像图像；
第一扩展单元，用于在第一扩展方向对所述区域图像进行扩展处理，获得至少一个第一 扩展图像；至少一个所述第一扩展图像包括所述区域图像；
第二扩展单元，用于在第二扩展方向对所述镜像图像进行扩展处理，获得至少一个第二扩展图像；所述第一扩展方向和所述第二扩展方向不同；
图像渲染单元，用于对所述背景图像、至少一个所述第二扩展图像和至少一个所述第一扩展图像进行渲染，获得渲染结束时的目标渲染图像；
视频合成单元，用于将所述图像帧对应的目标渲染图像进行视频合成，获得所述待处理视频对应的目标视频。
第三方面，本公开实施例提供一种电子设备，包括：处理器以及存储器；
所述存储器存储计算机执行指令；
所述处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如上第一方面以及第一方面各种可能的设计所述的视频合成方法。
第四方面，本公开实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能的设计所述的视频合成方法。
第五方面，本公开实施例提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的视频合成方法。
第六方面，本公开实施例还提供一种计算机程序，该计算机程序被处理器运行时实现如上第一方面以及第一方面各种可能的设计的视频合成方法。
附图说明
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种用于分身视频合成方法的一个应用示意图；
图2为本公开实施例提供的一种视频合成方法的一个实施例的流程图；
图3为本公开实施例提供的一个镜像图像的示例图；
图4为本公开实施例提供的一种视频合成方法的又一个实施例的流程图；
图5为本公开实施例提供的一个位移点的示例图；
图6为本公开实施例提供的一种视频合成方法的又一个实施例的流程图；
图7为本公开实施例提供的一个目标渲染图像的示例图；
图8为本公开实施例提供的一种视频合成方法的又一个实施例的流程图；
图9为本公开实施例提供的一种视频合成装置的一个结构示例图；
图10为本公开实施例提供的一种电子设备的硬件结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开 一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
本公开的技术方案可以应用于视频领域，可以通过对视频进行分身提取并扩展、渲染获得渲染图像，并利用各个视频帧的渲染图像合成视频，这一过程中从图像帧提取至图像分割、图像扩展以及渲染合成，均可以自动完成，获得渲染结束时的分身视频，提高分身视频的显示效率。
相关技术中，各种直播视频、短视频对视频的实时处理性需求较高。目前，较为常见的视频实时处理方式一般是，在视频中增加弹幕、贴图等内容，实现视频播放趣味性的提升。但是，这种处理方式较为单一，对视频的传播速度和推荐速度的提升并不高。
为了解决上述技术问题，本公开中考虑增加对视频中处于运动状态的对象进行分身，以增加视频为趣味性。因此，考虑对每帧图像进行自动分身，然后将分身后的视频帧合成为目标视频。为了获得分身的图像，可以将目标对象的区域图像和镜像图像在不同方向进行扩展，然后将扩展的图像进行统一渲染，获得对应的目标渲染图像。通过这种方式可以实现图像中的对象分身以利用视频合成，提高视频的合成效率和趣味性，对视频的播放率提升具有积极作用。
本公开的实施例中，对待处理视频中的图像帧进行分割，获得目标对象所在的区域图像和背景图像，在第一扩展方向对区域图像进行扩展和第二扩展方向对镜像图像进行图像扩展，获得至少一个第一扩展图像和至少一个第二扩展图像，通过至少一个第一扩展图像和至少一个第二扩展图像以及背景图像的渲染，获得具备目标对象分身的目标渲染图像。目标渲染图像是通过渲染背景图像、至少一个第二扩展图像和至少一个第一扩展图像获得的，可以展示正向的分身和反向的分身，提高图像的展示效率。将图像帧对应的目标渲染图像进行视频合成，获得待处理视频的目标视频，通过自动完成图像渲染和目标视频的自动合成，可以提高视频的获取效率和精度，确保目标视频的播放量和播放速度得到有效提升。
下面将以具体实施例对本公开的技术方案以及本公开的技术方案如何解决上述技术问题进行详细说明。下面几个具体实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图对本发明的实施例进行详细描述。
图1是根据本公开视频合成方法的一个应用示意图。根据本公开实施例的应用示意图中可以包括一个电子设备以及一个与该电子设备通过局域网或者广域网进行网络连接的用户端，假设该电子设备可以为个人计算机、普通服务器，超级个人计算机，云服务器等类型的服务器，本公开中对电子设备的具体类型并不作出过多限定。客户端例如可以为手机、平板电脑、个人计算机、智能家电、可穿戴设备等终端设备，本公中对客户端的具体类型并不作出过多限定。如图1所示，以电子设备为云服务器1，用户端2可以配置于手机21、平板电脑22等终端设备为例，任一个用户端2可以向云服务器1提供待处理视频。云服务器1中可以获取任一个用户端2发送的待处理视频，并基于本公开实施例提供的视频合成方法，合成目标视频。
之后，云服务器1可以将合成的目标视频发送至用户端2。此外，在实际应用中，目标视频被用户端2确认发布之后，可以对目标视频进行发布。目标视频发布成功，其它用户端3，例如图1所示的手机3，可以获取目标视频并播放目标视频。当然在实际应用中也可以向其它用户端3推荐目标视频，提高目标视频的播放率。
参考图2，图2为本公开实施例提供的一种视频合成方法的一个实施例的流程图，该方法可以配置为一视频合成装置，视频合成装置可以位于电子设备中。其中，视频合成方法可以包括以下几个步骤：
201：对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除区域图像之外的背景图像。
可选地，图像帧可以从待处理视频中采样获得。在实际应用中，待处理视频可以对应有帧率((Frame rate))＝帧数(Frames)/时间(Time)，帧率的单位为帧每秒(f/s,frames per second,fps)，在从待处理视频中采样时，可设置采样频率包括待处理视频的帧率。若设置采样频率与图像帧的帧率相同，则可以确保视频中的每个图像帧均可以被采样，并采用本公开的技术方案进行图像处理以获得对应的目标渲染图像。
当然，在实际应用中，待处理视频的采样频率也可以根据实际需求设置，并不局限于与帧率等同。采样频率越高，单位时间内被采集的图像帧的数量越多，出现不连续画面的可能性越低。采样频率越低，单位时间内被采集的图像着呢的数量越少，出现不连续画面的可能性越高。
图像帧可以从待处理视频中采样获得。图像帧可以包括至少一个，每个图像帧均可以通过步骤201-204获得对应的目标渲染图像。
区域图像可以为目标对象的轮廓所在区域的图像，可以为不规则的图像。背景图像可以为待处理视频中除目标对象轮廓之外的其它区域所对应的图像。
可选地，目标对象可以包括人体、物体、动物等可以运动的物体，较为常见的目标对象例如可以包括运动过程中的人或者动物。
202：确定区域图像对应的镜像图像。
镜像图像可以通过对区域图像进行镜像翻转获得，镜像图像的图像内容与区域图像相同，但是显示方向相反。
镜像图像可以包括水平镜像图像、垂直镜像图像和对角镜像图像中的任一种。其中，水面镜像图像是指将图像的左右部分以图像垂直中轴线为中心进行镜像对换。垂直镜像图像可以是将图像的上下两个部分以图像水平中轴线为中心进行镜像对换。而对角镜像图像可以是将图像以图像水平中轴线和垂直中轴线的交点为中心进行镜像对换，相当于将图像先后执行水平镜像和垂直镜像。为了便于理解，参考图3所示的三角形301，其顶点A在边长B的左边，经过水平镜像对换301之后，顶点A’在边长B’的右边。
203：在第一扩展方向对区域图像进行扩展处理，获得至少一个第一扩展图像；至少一个第一扩展图像包括区域图像。
204：在第二扩展方向对镜像图像进行扩展处理，获得至少一个第二扩展图像。
其中，第一扩展方向和第二扩展方向不同。
在某些实施例中，第一扩展方向和第二扩展方向可以相反。也即，第一扩展方向和第二扩展方向的夹角为180度。当然，第一扩展方向和第二扩展方向也可以是基于水平线的垂线所形成的两个夹角确定，水平线的垂线可以作为两个夹角的公共边，两个夹角的另一边指向的方向可以分别为第一扩展方向和第二扩展方向。假设两个夹角使用夹角A和夹角B表示。在实际应用中夹角A和夹角B的夹角角度可以相等也可以不相等，夹角A和夹角B的夹角角度在0至180度之间。其中，夹角为90度时，第一扩展方向和第二扩展方向相反。夹角A和 夹角B相同时，第一扩展方向和第二扩展方向相对于水平线的垂线对称。
可选地，以在水平线的垂线为例，第一扩展方向可以包括水平线的垂线右侧，第二扩展方向可以包括水平线的垂线右侧。在实际应用中，水平线可以包括目标对象的中心点所对应的水平线。为了便于理解，以图像在显示屏幕显示，人观看显示屏幕中的图像为例，人眼能观看到的显示屏幕中图像的左方可以为反向，人眼能观看到的显示屏幕中图像的右方可以为正向。正向和反向具体可以是指图像的左方扩展还是右方扩展。区域图像可以向右进行第一扩展方向的扩展，获得至少一个第一扩展图像，向左进行第二扩展方向的扩展，获得至少一个第二扩展图像。
205：对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像。
可选地，目标渲染图像可以为背景图像、至少一个第二扩展图像和至少一个第一扩展图像分别渲染获得的，具体可以包括背景图像、至少一个第二扩展图像和至少一个第一扩展图像多层渲染结束时获得的渲染图像。
206：将图像帧对应的目标渲染图像进行视频合成，获得待处理视频对应的目标视频。
目标视频可以由多个图像帧分别对应的目标渲染图像合成获得。将图像帧对应的目标渲染图像进行视频合成时，可以按照图像帧的时间戳将该图像帧对应的目标渲染图像进行视频合成。图像帧的时间戳即为其对应的目标渲染图像的时间戳。视频合成的过程中，可以根据待处理视频的帧率确定目标视频的帧率，通过视频的帧率可以对视频的合成时间进行准确控制。
本公开实施例中，对待处理视频中的图像帧进行分割，获得目标对象所在的区域图像和背景图像，利用区域图像和区域图像的镜像图像进行图像扩展，获得至少一个第一扩展图像和至少一个第二扩展图像，通过至少一个第一扩展图像和至少一个第二扩展图像以及背景图像的渲染，获得具备目标对象分身的目标渲染图像。目标渲染图像是通过渲染背景图像、至少一个第二扩展图像和至少一个第一扩展图像获得的，可以展示正向的分身和反向的分身，提高图像的展示效率。之后可以将图像帧对应的目标渲染图像进行视频合成，获得待处理视频的目标视频，通过自动完成图像渲染和目标视频的自动合成，可以提高视频的获取效率和精度，确保目标视频的播放量和播放速度得到有效提升。
此外，本公开中涉及的待处理视频可以是实时采集的，通过对实时采集的视频进行图像帧的提取。之后对图像帧进行目标对象的区域分割，以利用分割获得的区域图像进行镜像翻转，实现正向的区域图像的扩展和镜像的镜像图像的扩展，获得至少一个第一扩展图像和至少一个第二扩展图像。通过对除区域图像之外的背景图像、至少以第一扩展图像和至少一个第二扩展图像的渲染，获得目标渲染图像，以利用目标渲染图像进行视频的合成。这一过程中均是自动完成的，并不需要人工介入，例如常见的绘图软件的手动复制等操作，视频的合成效率较高，可以解决现有技术中手动完成分身视频的合成效率较低的问题。
本实施例中，对待处理视频中的图像帧进行分割，获得目标对象所在的区域图像和背景图像，利用区域图像和区域图像的镜像图像进行图像扩展，具体可以通过第一扩展方向对区域图像进行扩展，获得至少一个第一扩展图像，通过第二扩展方向对镜像图像进行扩展，获得至少一个第二扩展图像。通过图像的扩展，可以实现对目标对象的分身。进而通过至少一个第一扩展图像和至少一个第二扩展图像以及背景图像的渲染，获得具备目标对象分身的目 标渲染图像。目标渲染图像是通过渲染背景图像、至少一个第二扩展图像和至少一个第一扩展图像获得的，可以展示第一扩展方向和第二扩展方向的扩展图像，提高图像的展示效率和趣味性。将图像帧对应的目标渲染图像进行视频合成，获得待处理视频的目标视频，通过自动完成图像渲染和目标视频的自动合成，可以提高视频的获取效率和精度，确保目标视频的播放速度和效率得到有效提升。
为了获得准确的第一扩展图像，如图4所示，为本公开实施例提供的一种视频合成方法的又一个实施例的流程图，与其它实施例的不同之处在于，其中，在第一扩展方向对区域图像进行扩展处理，获得至少一个第一扩展图像，可以包括：
401：确定目标对象在图像帧对应的位移速度。
位移速度可以指目标对象在待处理视频中的移动速度。例如假设目标对象从视频播放时，在N位置，在播放至第二帧图像时，移动至M位置，则目标对象的位移速度可以根据N位置和M位置的差值结合两个图像帧的时间差计算获得。
402：以区域图像的原始位置点作为基础，按照位移速度和第一扩展方向进行位移处理，获得包含原始位置点的至少一个正向位移点。
区域图像可以指将图像帧中属于目标对象的像素点进行标记，获得区域图像在图像帧中的像素点，区域图像所对应像素点的坐标点还可以通过图像帧的像素坐标系表示。未被标记为区域图像的像素点即为背景图像的像素点。
其中，区域图像的原始位置点可以指目标对象的关键点在图像帧所对应的像素坐标点。关键点可以指能够对区域图像的位置进行确认的像素点，该像素点的坐标是通过整个图像帧的像素坐标系表示的。原始位置点例如可以包括区域图像的中心点在图像帧中的坐标点。例如，目标对象为人体为例，还可以为人体中某个关键位置点，例如眉心点、鼻尖点、中心点等像素点的坐标，关于原始位置点所对应的位置含义并不在此进行限制。同样，正向位移点可以包括原始位置点产生位移之后获得的位于图像帧中的像素坐标点。原始位置点和正向位移点均可以包括位于图像帧的像素坐标系中的坐标点。
可选地，第一扩展方向可以包括正向位移角度。第一扩展方向可以通过目标对象在视频帧中的轨迹确定。例如，同样以由N位置移动到M位置为例，两个位置连线相对于水平线或者垂线可能存在夹角，该夹角即为正向位移角度。假设正向位移角度为45度为例，目标对象位移时，可以向右上方且与水平线夹角为45度的直线，移动的距离按照位移速度计算获得点，即为一个正向位移点。为了便于理解，如图5所示，假设P0为原始位置点，P0按照正向位移45度，获得的正向位移点可以为P1。
403：将区域图像复制到正向位移点，获得区域图像在至少一个正向位移点分别对应的第一扩展图像。
将区域图像复制到正向位移点，可以包括将区域图像的像素点所对应的像素值，按照正向位移点移动到对应的坐标点，也即，可以根据区域图像在图像帧中的坐标点和正向位移点，确定第一扩展图像在图像帧中的正向坐标点，通过将区域图像中的坐标点处的像素值赋值给对应的正向坐标点，即可以获得正向坐标点所对应的像素值构成的第一扩展图像。当然，在实际应用中，区域图像的像素点可以包括多个，可以根据正向位移点一一将每个像素点进行移动，获得移动后的多个像素点组成的第一扩展图像。第一扩展图像中，除移动后的区域图像的像素点的坐标值为真，其它像素点的坐标被设置为透明属性，显示时均是透明的，不存 在颜色的，以不对其它图像，例如背景图像产生显示遮挡。
正向位移点可以包括原始位置点，也即在不对区域图像进行位移的情况下，生成一原始位置点对应的第一扩展图像，可以确保原始位置处的区域图像能够正常显示。原始位置点对应的第一扩展图像的像素点的坐标不发生变化。
本公开实施例，可以通过获取目标对象的位移速度，在目标对象的原始位置点的基础上进行正向位移获得准确的至少一个正向位移点。以利用至少一个正向位移点，将区域图像按照正向位移点进行确认，以获得正向位移点确定对应的第一扩展图像，提高第一扩展图像的定位准确性，获得定位精度更高的至少一个正向位移点分别对应的第一扩展图像。
在一种可能的设计中，以区域图像的原始位置点作为基础，按照位移速度和第一扩展方向进行位移处理，获得包含原始位置点的至少一个正向位移点，可以包括：
确定区域图像的区域宽度和图像帧的图像宽度。
基于区域宽度和图像宽度，结合位移速度，在第一扩展方向上计算区域图像的正向位移数量以及正向偏移步长。
根据区域图像的原始位置，按照每次正向偏移步长和正向位移数量，确定至少一个正向位置点，获得原始位置点和至少一个正向位置点构成的至少一个正向位移点。
可选地，区域图像的区域宽度可以指对象轮廓区域的水平宽度。图像宽度也可以指图像帧的水平宽度。
正向位移数量可以根据区域宽度和图像宽度自动计算获得，例如可以计算二分之一的图像宽度和区域宽度的商，再将获得的结果取整，即可以获得正向位移数量。但是，在实际应用中，正向位移数量也可以预先设置获得。例如可以由运维人员设置也可以由使用该视频合成方法的前端用户设置获得。在较为常见的场景中可以设置正向位移数量为3。
正向偏移步长可以根据二分之一图像宽度、区域宽度的宽度差值和正向位移数量求商，以位移速度作为约束条件，确定的。通常可以计算宽度差值和正向位移数量的商，再将求商结果取整即可以获得初始的正向偏移步长，若位移速度大于某个阈值，则步长减小，若位移速度小于某个阈值，则步长增加。
本公开实施例中，确定正向位移点时，可以根据区域图像的区域宽度和图像帧的图像宽度准确计算第一扩展方向上的正向位移数量和正向偏移步长，可以通过正向位移数量和正向偏移步长，准确对正向位置点进行定位，获得位置精度更高的正向位移点。
为了获得准确的第二扩展图像，如图6所示，为本公开实施例提供的一种视频合成方法的又一个实施例的流程图，与其它实施例的不同之处在于，其中，在第二扩展方向对镜像图像进行扩展处理，获得至少一个第二扩展图像，包括：
601：确定目标对象在图像帧对应的位移速度。
本实施例中部分步骤与其它实施例中部分步骤相同，在此不再赘述。
602：以区域图像的原始位置点作为基础，基于位移速度和第二扩展方向进行位移处理，获得与原始位置点相对应的至少一个反向位移点。
区域图像可以指将图像帧中属于目标对象的像素点进行标记，获得区域图像在图像帧中的像素点，区域图像所对应像素点的坐标点还可以通过图像帧的像素坐标系表示。未被标记为区域图像的像素点即为背景图像的像素点。
其中，区域图像的原始位置点可以指目标对象的关键点在图像帧所对应的像素坐标点。 关键点可以指能够对区域图像的位置进行确认的像素点，该像素点的坐标是通过整个图像帧的像素坐标系表示的。原始位置点例如可以包括区域图像的中心点在图像帧中的坐标点。例如，目标对象为人体为例，还可以为人体中某个关键位置点，例如眉心点、鼻尖点、中心点等像素点的坐标，关于原始位置点所对应的位置含义并不在此进行限制。同样，反向位移点可以包括原始位置点产生位移之后获得的位于图像帧中的像素坐标点。原始位置点和反向位移点均可以包括位于图像帧的像素坐标系中的坐标点。
可选地，第二扩展方向可以包括反向位移角度。第二扩展方向可以通过目标对象在视频帧中的轨迹确定。例如，同样以由N位置移动到M位置为例，两个位置连线相对于水平线或者垂线可能存在夹角，该夹角即为反向位移角度。反向位移过程与正向位移类似，在此不再赘述。
603：将镜像图像复制到反向位移点，获得镜像图像在至少一个反向位移点分别对应的第二扩展图像。
将区域图像复制到反向位移点，可以包括将区域图像的像素点所对应的像素值，按照反向位移点移动到对应的坐标点，也即，可以根据区域图像在图像帧中的坐标点和反向位移点，确定第二扩展图像在图像帧中的反向坐标点，通过将区域图像中的坐标点处的像素值赋值给对应的反向坐标点，即可以获得反向坐标点所对应的像素值构成的第二扩展图像。当然，在实际应用中，区域图像的像素点可以包括多个，可以根据反向位移点一一将每个像素点进行移动，获得移动后的多个像素点组成的第二扩展图像。第二扩展图像中，除移动后的区域图像的像素点的坐标值为真，其它像素点的坐标被设置为透明属性，显示时均是透明的，不存在颜色的，以不对其它图像，例如背景图像产生显示遮挡。
本公开实施例中，可以通过获取目标对象的位移速度，在目标对象的原始位置点的基础上进行反向位移获得准确的至少一个反向位移点。以利用至少一个反向位移点，将区域图像按照反向位移点进行确认，以获得反向位移点确定对应的第二扩展图像，提高第二扩展图像的定位准确性，获得定位精度更高的至少一个反向位移点分别对应的第二扩展图像。
作为一个实施例，以区域图像的原始位置点作为基础，基于位移速度和第二扩展方向进行位移处理，获得与原始位置点相对应的至少一个反向位移点，包括：
确定区域图像的区域宽度和图像帧的图像宽度；
基于区域宽度和图像宽度，结合位移速度，在第二扩展方向上计算区域图像的反向位移数量以及反向偏移步长；
根据区域图像的原始位置，按照反向偏移步长和反向偏移数量，确定至少一个反向位置点，获得至少一个反向位置点构成的至少一个反向位移点。
可选地，区域图像的区域宽度可以指对象轮廓区域的水平宽度。图像宽度也可以指图像帧的水平宽度。
反向位移数量可以根据区域宽度和图像宽度自动计算获得，例如可以计算二分之一的图像宽度和区域宽度的商，再将获得的结果取整，即可以获得反向位移数量。但是，在实际应用中，反向位移数量也可以预先设置获得。例如可以由运维人员设置也可以由使用该视频合成方法的前端用户设置获得。在较为常见的场景中可以设置反向位移数量为3。
反向偏移步长可以根据二分之一图像宽度、区域宽度的宽度差值和反向位移数量求商，以位移速度作为约束条件，确定的。通常可以计算宽度差值和反向位移数量的商，再将求商 结果取整即可以获得初始的反向偏移步长，若位移速度大于某个阈值，则步长减小，若位移速度小于某个阈值，则步长增加。
本公开实施例中，确定反向位移点时，可以根据区域图像的区域宽度和图像帧的图像宽度准确计算第二扩展方向上的反向位移数量和反向偏移步长，可以通过反向位移数量和反向偏移步长，准确对反向位置点进行定位，获得位置精度更高的反向位移点。
作为一个实施例，确定目标对象在图像帧对应的位移速度，包括：
确定图像帧在待处理视频对应的前一个图像帧。
确定目标对象在前一个图像帧的第一位置和在图像帧的第二位置；
根据第一位置和第二位置，计算目标对象对应的位移量；
利用待处理视频的图像帧显示时间间隔，和位移量，计算获得目标对象在图像帧对应的位移速度。
可选地，根据第一位置和第二位置，计算目标对象对应的位移量可以包括：确定待处理视频的帧率，根据待处理视频的帧率计算两个图像帧之间的时间间隔，根据第一位置和第二位置计算位置差值。该位置差值可以指第一位置和第二位置的绝对差值，具体可以通过两个位置的横向位移差值和纵向位移差值进行三角形计算获得，也即以横向差值和纵向差值分别坐标直角三角形的两个直角边，位移量为该直角三角形的斜边长度。位移量和时间间隔的商即可以为位移速度。
本公开实施例中，可以通过获取前一个图像帧中目标对象所在的第一位置，和当前图像帧的第二位置，通过计算第一位置和第二位置，可以计算目标对象对应的位移量，通过位移量和两个图像帧之间的时间间隔，可以准确计算目标图像的位移速度。通过相邻两个图像帧之间的位移量和时间间隔可以对目标对象的位移速度进行准确计算，获得高精度的位移速度。
为了确保图像显示的准确性，使得用户能够正常观看到目标对象，在一种可能的设计中，对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像，可以包括：
按照背景图像为第一渲染顺序，至少一个第二扩展图像为第二渲染顺序和至少一个第一扩展图像为第三渲染顺序，依次叠加渲染背景图像、至少一个第二扩展图像和至少一个第一扩展图像，获得渲染结束时的目标渲染图像。
可选地，对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像可以包括：对背景图像、至少一个第二扩展图像和至少一个第一扩展图像分别进行渲染，获得渲染结束时的目标渲染图像。目标渲染图像可以由背景图像、至少一个第二扩展图像和至少一个第一扩展图像分别渲染获得。
本公开实施例中，可以分别为背景图像、至少一个第二扩展图像和至少一个第一扩展图像分别设置渲染顺序，以实现目标渲染图像的准确渲染，避免因图像间的随机渲染导致图像显示覆盖问题，确保显示的有效性。
作为一种可能的实现方式，依次叠加渲染背景图像、至少一个第二扩展图像和至少一个第一扩展图像，获得渲染结束时的目标渲染图像，可以包括：
将背景图像在最底层渲染。
确定至少一个第二扩展图像与背景图像的中心点分别对应的反向中心距离；
按照反向中心距离从大到小的顺序，依次渲染至少一个第二扩展图像；
确定至少一个第一扩展图像与背景图像的中心点分别对应的正向中心距离；
按照正向中心距离从大到小的顺序，依次渲染至少一个第一扩展图像，直至最后一个第一扩展图像渲染结束，获得目标渲染图像。
第二扩展图像与背景图像中心点的反向中心距离可以通过第二扩展图像的反向位移点与背景图像的中心点计算获得。同样，第一扩展图像与背景图像的中心点的正向中心距离，可以通过第一扩展图像的正向位移点和背景图像的中心点计算获得。
目标渲染图像可以为背景图像、至少一个第二扩展图像和至少一个第一扩展图像多层渲染获得。具体可以是将背景图像在最底层渲染，将第二扩展图像在背景图像的上层渲染，将第一扩展图像在第二扩展图像的上层渲染。而至少一个第二扩展图像渲染过程中也存在渲染顺序，也即将距离图像中心点距离最大的第二扩展图像先渲染，随着距离的减少，渲染顺序越靠后。至少一个第一扩展图像渲染过程相似，也即将距图像中心点距离最大的第一扩展图像先渲染，随着距离的减少，渲染顺序越靠后。
当然，还可以使用其他的渲染顺序。例如至少一个第二扩展图像和至少一个第一扩展图像同批次渲染，直接按照距离中心点的距离的大小，从最大的距离对应的图像开始渲染，直至最后一个第一扩展图像渲染结束。第一扩展图像包括区域图像，区域图像距离中心点的位置最小。为了便于理解，如图7所示，假设区域图像为701所示，则对区域图像进行第一扩展方向的扩展处理和第二扩展方向的扩展处理后，在显示屏幕700中显示的至少一个第一扩展图像分别为701-703，至少一个第二扩展图像分别为704-705。
本公开实施例中，将背景图像在最底层渲染，确保背景图像正常显示。第一扩展图像和第二扩展图像可以位于背景图像的上层渲染，在渲染正第二扩展图像时，可以按照反向中心距离和正向中心距离的从大到小的顺序，依次将第二扩展图像和第一扩展图像向上层渲染，使得图像的渲染层次与距离相关，实现图像的准确渲染，确保区域图像能够在最上层渲染，获得精准渲染的目标渲染图像。
如图8所示，为本公开实施例提供的一种视频合成方法的又一个实施例的流程图，该视频合成方法中可以为区域图像设置贴图图像，参考图8，对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像之后，本公开中的视频合成方法还可以包括：
801：获取用户为目标对象选择的贴图图像。
802：基于区域图像在图像帧的原始位置点，确定贴图图像在图像帧中的贴图位置。
803：将贴图图像按照贴图位置渲染至目标渲染图像中，获得目标贴图图像。
804：将图像帧对应的目标贴图图像进行视频合成，获得待处理视频对应的贴图目标视频。
贴图图像可以为提供待处理视频的用户指定，可以指各种卡通形象、风景图像、大头贴等诸多类型的贴图。当然，贴图位置也可以由该用户指定。根据原始位置点可以对贴图位置点进行定位。
可以基于区域图像在图像帧的原始位置点，确定区域图像中头部顶点所在的位置点，将该头部顶点所在的位置点作为贴图位置。
本公开实施例中，在获得用户为目标对象选择的贴图图像之后，可以基于区域图像在图像帧的原始位置点，确定贴图图像在图像帧中的贴图位置。通过贴图位置的确定，可以将贴图图像按照贴图位置渲染至目标渲染图像中，获得目标贴图图像，将图像帧对应的目标贴图 图像进行视频合成，即可以获得待处理视频对应的贴图目标视频。通过贴图可以使得目标视频的内容更全面化，提高视频显示效率。
作为一个实施例，对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除区域图像之外的背景图像，包括：
从待处理视频中提取图像帧；
识别图像帧中目标对象的对象轮廓区域；
利用对象轮廓区域分割图像帧，获得对象轮廓区域对应的区域图像和除区域图像之外的背景图像。
可选地，可以采用轮廓提取算法识别图像帧中目标对象的对象轮廓区域。轮廓提取算法可以为常见的轮廓提取算法，本公开中对轮廓提取算法的具体类型并不作出过多限定。图像分割具体可以指将对象轮廓区域所包围的像素点设置为对象标识，将位于对象轮廓区域之外的其它像素点设置为背景标识。可以通过像素点的标识，区分区域图像和背景图像的像素点。
本公开实施例中，可以从待处理视频中提取图像帧，并识别图像帧中目标对象的对象轮廓区域，对象轮廓区域可以用于对图像帧进行分割，以获得准确的对象轮廓区域的区域图像和背景图像。通过图像分割可以实现图像的准确分割，实现区域图像和背景图像的准确获取。
在一种可能的设计中，该方法还可以包括：
基于第一用户触发的目标视频采集请求，获取待处理视频。
将图像帧对应的目标渲染图像进行视频合成，获得待处理视频对应的目标视频，之后，还包括：
为第一用户输出目标视频。
待处理视频可以实时采集，也可以从已存储的视频中读取获得。
其中，为第一用户输出目标视频可以包括利用输出装置为第一用户输出目标视频或者将目标视频发送至第一用户对应的用户设备，由用户设备中的输出装置输出目标视频。
本公开实施例中，可以获取第一用户提供的待处理视频，在获得目标视频之后，还可以为第一用户输出目标视频。通过与第一用户交互处理可以实现待处理视频的获取和目标视频的输出，以便于为第一用户以更高效率、更自动化模式提供目标视频。
作为一个实施例，为第一用户输出目标视频之后，还包括：
检测第一用户针对目标视频触发的发布请求；
响应于发布请求，将目标视频发布至第一用户的用户视频账户，目标视频供第二用户的第二用户设备读取并播放。
可选地，电子设备可以为第一用户提供目标视频的发布控件。第一用户的用户设备检测到第一用户触发发布控件时，用户设备即可以发送发布请求至电子设备。电子设备可以响应于该发布请求以将目标视频存储至第一用户的用户视频账户中，具体可以是将目标视频存储，并建立该目标视频的视频名称或者视频标识与第一用户的账户名称之间建立存储关联。用户视频账户可以对应多个视频，其中可以包括目标视频。第二用户设备需要播放目标视频时，从第一用户的用户视频账户中确定身份视频，将目标视频发送至第二用户设备。其中，目标视频被播放一次，该目标视频的播放次数随之增加。
本公开实施例中，可以检测第一用户针对目标视频触发的发布请求，将目标视频发布至第一用户的用户视频账户，目标视频可以供第二用户的第二用户设备读取并播放。通过第一 用户对目标视频的发布控制，可以实现目标视频的有效发布。
在某些实施例中，检测第一用户触发的目标视频采集请求，获取待处理视频包括：
接收第一用户设备发送的待处理视频；待处理视频是第一用户的用户设备在用户触发目标视频采集请求时采集获得的；
为第一用户输出目标视频，包括：
发送目标视频至第一用户设备，并控制第一用户设备播放目标视频。
当然，在某些实施例中，电子设备可以直接与第一用户进行交互，采集待处理视频，并基于本公开的视频合成方法合成待处理视频对应的目标视频，实现在单个电子设备中完成视频的采集和处理，提高电子设备的利用率。
本公开实施例中，可以接收第一用户设备发送的待处理视频，待处理视频可以为第一用户的用户设备采集。通过为待处理视频的视频帧进行目标视频的自动生成，可以将生成的目标视频发送至第一用户设备，实现向第一用户设备反馈目标视频。实现目标视频的生成和输出的分离，降低第一用户设备的处理压力，对第一用户设备的正常运行起到保障作用。同时，利用运算能力更强的电子设备完成目标视频的生成，可以使得目标视频的生成效率和稳定性更高，确保目标视频的生成效率和成功率。
如图9所示，为本公开实施例提供的一种视频合成装置的一个实施例的结构示意图，该视频合成装置900可以包括：
图像分割单元901：用于对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除区域图像之外的背景图像；
镜像处理单元902：用于确定区域图像对应的镜像图像；
第一扩展单元903：用于在第一扩展方向对区域图像进行扩展处理，获得至少一个第一扩展图像；至少一个第一扩展图像包括区域图像；
第二扩展单元904：用于在第二扩展方向对镜像图像进行扩展处理，获得至少一个第二扩展图像；第一扩展方向和第二扩展方向不同；
图像渲染单元905：用于对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像；
视频合成单元906：用于将图像帧对应的目标渲染图像进行视频合成，获得待处理视频对应的目标视频。
作为一个实施例，第一扩展单元，可以包括：
速度确定模块，用于确定目标对象在图像帧对应的位移速度；
第一扩展模块，用于以区域图像的原始位置点作为基础，按照位移速度和第一扩展方向进行位移处理，获得包含原始位置点的至少一个正向位移点；
第一确定模块，用于将区域图像复制到正向位移点，获得区域图像在至少一个正向位移点分别对应的第一扩展图像。
在某些实施例中，第一扩展模块，可以包括：
第一确定子模块，用于确定区域图像的区域宽度和图像帧的图像宽度；
第一计算子模块，用于基于区域宽度和图像宽度，结合位移速度，在第一扩展方向上计算区域图像的正向位移数量以及正向偏移步长；
正向位移子模块，用于根据区域图像的原始位置，按照正向偏移步长和正向位移数量， 确定至少一个正向位置点，获得原始位置点和至少一个正向位置点构成的至少一个正向位移点。
作为一个实施例，第二扩展单元，可以包括：
速度确定模块，用于确定目标对象在图像帧对应的位移速度；
第二扩展模块，用于以区域图像的原始位置点作为基础，基于位移速度和第二扩展方向进行位移处理，获得与原始位置点相对应的至少一个反向位移点；
第二确定模块，用于将镜像图像复制到反向位移点，获得镜像图像在至少一个反向位移点分别对应的第二扩展图像。
在某些实施例中，第二扩展模块，包括：
第二确定子模块，用于确定区域图像的区域宽度和图像帧的图像宽度；
第二计算子模块，用于基于区域宽度和图像宽度，结合位移速度，在第二扩展方向上计算区域图像的反向位移数量以及反向偏移步长；
反向位移子模块，用于根据区域图像的原始位置，按照反向偏移步长和反向偏移数量，确定至少一个反向位置点，获得至少一个反向位置点构成的至少一反向位移点。
在某些实施例中，速度确定模块，包括：
第三确定模块，用于确定图像帧在待处理视频对应的前一个图像帧；
第四确定模块，用于确定目标对象在前一个图像帧的第一位置和在图像帧的第二位置；
位移计算模块，用于根据第一位置和第二位置，计算目标对象对应的位移量；
速度计算模块，用于利用待处理视频的图像帧显示时间间隔，和位移量，计算获得目标对象在图像帧对应的位移速度。
在某些实施例中，图像渲染单元，可以包括：
顺序渲染模块，用于按照背景图像为第一渲染顺序，至少一个第二扩展图像为第二渲染顺序和至少一个第一扩展图像为第三渲染顺序，依次叠加渲染背景图像、至少一个第二扩展图像和至少一个第一扩展图像，获得渲染结束时的目标渲染图像。
在某些实施例中，顺序渲染模块，包括：
第一渲染模块，用于将背景图像在最底层渲染；
第一距离模块，用于确定至少一个第二扩展图像与背景图像的中心点分别对应的反向中心距离；
第二渲染模块，用于按照反向中心距离从大到小的顺序，依次渲染至少一个第二扩展图像；
第二距离模块，用于确定至少一个第一扩展图像与背景图像的中心点分别对应的正向中心距离；
第二渲染模块，用于按照正向中心距离从大到小的顺序，依次渲染至少一个第一扩展图像，直至最后一个第一扩展图像渲染结束，获得目标渲染图像。
作为又一个实施例，该装置还包括：
贴图获取单元，用于获取用户为目标对象选择的贴图图像；
位置确定单元，用于基于区域图像在图像帧的原始位置点，确定贴图图像在图像帧中的贴图位置；
图像贴图单元，用于将贴图图像按照贴图位置渲染至目标渲染图像中，获得目标贴图图 像；
贴图视频单元，用于将图像帧对应的目标贴图图像进行视频合成，获得待处理视频对应的贴图目标视频。
在某些实施例中，图像分割单元，包括：
图像提取模块，用于从待处理视频中提取图像帧；
区域识别模块，用于识别图像帧中目标对象的对象轮廓区域；
图像分割模块，用于利用对象轮廓区域分割图像帧，获得对象轮廓区域对应的区域图像和除区域图像之外的背景图像。
在某些实施例中，该装置还包括：
视频获取单元，用于基于第一用户触发的目标视频采集请求，获取待处理视频；
视频输出单元，用于为第一用户输出目标视频。
作为又一个实施例，该装置还可以包括：
发布检测单元，用于检测第一用户针对目标视频触发的发布请求；
请求响应单元，用于响应于发布请求，将目标视频发布至第一用户的用户视频账户，目标视频供第二用户的第二用户设备读取并播放。
在某些实施例中，视频获取单元，包括：
视频接收模块，用于接收第一用户设备发送的待处理视频；待处理视频是第一用户的用户设备在用户触发目标视频采集请求时采集获得的；
视频输出单元，包括：
视频发送模块，用于发送目标视频至第一用户设备，并控制第一用户设备播放目标视频。
本实施例提供的装置，可用于执行上述方法实施例的技术方案，其实现原理和技术效果类似，本实施例此处不再赘述。
为了实现上述实施例，本公开实施例还提供了一种电子设备。
参考10，其示出了适于用来实现本公开实施例的电子设备1000的结构示意图，该电子设备1000可以为终端设备或服务器。其中，终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，简称PDA)、平板电脑(Portable Android Device，简称PAD)、便携式多媒体播放器(Portable Media Player，简称PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图10示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图10所示，电子设备1000可以包括处理装置(例如中央处理器、图形处理器等)1001，其可以根据存储在只读存储器(Read Only Memory，简称ROM)1002中的程序或者从存储装置1008加载到随机访问存储器(Random Access Memory，简称RAM)1003中的程序而执行各种适当的动作和处理。在RAM 1003中，还存储有电子设备1000操作所需的各种程序和数据。处理装置1001、ROM 1002以及RAM 1003通过总线1004彼此相连。输入/输出(I/O)接口1005也连接至总线1004。
通常，以下装置可以连接至I/O接口1005：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置1006；包括例如液晶显示器(Liquid Crystal Display，简称LCD)、扬声器、振动器等的输出装置1007；包括例如磁带、硬盘等的存储装置1008；以及通信装置1009。通信装置1009可以允许电子设备1000与其他设备进行无线 或有线通信以交换数据。虽然图10示出了具有各种装置的电子设备1000，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置1009从网络上被下载和安装，或者从存储装置1008被安装，或者从ROM 1002被安装。在该计算机程序被处理装置1001执行时，执行本公开实施例的方法中限定的上述功能。
本公开的实施例还包括一种计算机程序，该计算机程序被处理器运行时执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备执行上述实施例所示的方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network，简称LAN)或广域网(Wide Area Network，简称WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个 模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
第一方面，根据本公开的一个或多个实施例，提供了一种目标视频的合成方法，包括：
对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除区域图像之外的背景图像；
确定区域图像对应的镜像图像；
在第一扩展方向对区域图像进行扩展处理，获得至少一个第一扩展图像；至少一个第一扩展图像包括区域图像；
在第二扩展方向对镜像图像进行扩展处理，获得至少一个第二扩展图像；第一扩展方向和第二扩展方向不同。
对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像；
将图像帧对应的目标渲染图像进行视频合成，获得待处理视频对应的目标视频。
根据本公开的一个或多个实施例，在第一扩展方向对区域图像进行扩展处理，获得至少一个第一扩展图像，包括：
确定目标对象在图像帧对应的位移速度；
以区域图像的原始位置点作为基础，按照位移速度和第一扩展方向进行位移处理，获得包含原始位置点的至少一个正向位移点；
将区域图像复制到正向位移点，获得区域图像在至少一个正向位移点分别对应的第一扩展图像。
根据本公开的一个或多个实施例，以区域图像的原始位置点作为基础，按照位移速度和第一扩展方向进行位移处理，获得包含原始位置点的至少一个正向位移点包括：
确定区域图像的区域宽度和图像帧的图像宽度；
基于区域宽度和图像宽度，结合位移速度，在第一扩展方向上计算区域图像的正向位移数量以及正向偏移步长；
根据区域图像的原始位置，按照正向偏移步长和正向位移数量，确定至少一个正向位置点，获得原始位置点和至少一个正向位置点构成的至少一个正向位移点。
根据本公开的一个或多个实施例，在第二扩展方向对镜像图像进行扩展处理，获得至少一个第二扩展图像，包括：
确定目标对象在图像帧对应的位移速度；
以区域图像的原始位置点作为基础，基于位移速度和第二扩展方向进行位移处理，获得与原始位置点相对应的至少一个反向位移点；
将镜像图像复制到反向位移点，获得镜像图像在至少一个反向位移点分别对应的第二扩展图像。
根据本公开的一个或多个实施例，以区域图像的原始位置点作为基础，基于位移速度和第二扩展方向进行位移处理，获得与原始位置点相对应的至少一个反向位移点，包括：
确定区域图像的区域宽度和图像帧的图像宽度；
基于区域宽度和图像宽度，结合位移速度，在第二扩展方向上计算区域图像的反向位移数量以及反向偏移步长；
根据区域图像的原始位置，按照反向偏移步长和反向偏移数量，确定至少一个反向位置点，获得至少一个反向位置点构成的至少一反向位移点。
根据本公开的一个或多个实施例，确定目标对象在图像帧对应的位移速度，包括：
确定图像帧在待处理视频对应的前一个图像帧；
确定目标对象在前一个图像帧的第一位置和在图像帧的第二位置；
根据第一位置和第二位置，计算目标对象对应的位移量；
利用待处理视频的图像帧显示时间间隔，和位移量，计算获得目标对象在图像帧对应的位移速度。
根据本公开的一个或多个实施例，对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像，包括：
按照背景图像为第一渲染顺序，至少一个第二扩展图像为第二渲染顺序和至少一个第一扩展图像为第三渲染顺序，依次叠加渲染背景图像、至少一个第二扩展图像和至少一个第一扩展图像，获得渲染结束时的目标渲染图像。
根据本公开的一个或多个实施例，依次叠加渲染背景图像、至少一个第二扩展图像和至少一个第一扩展图像，获得渲染结束时的目标渲染图像，包括：
将背景图像在最底层渲染；
确定至少一个第二扩展图像与背景图像的中心点分别对应的反向中心距离；
按照反向中心距离从大到小的顺序，依次渲染至少一个第二扩展图像；
确定至少一个第一扩展图像与背景图像的中心点分别对应的正向中心距离；
按照正向中心距离从大到小的顺序，依次渲染至少一个第一扩展图像，直至最后一个第 一扩展图像渲染结束，获得目标渲染图像。
根据本公开的一个或多个实施例，对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像之后，方法还包括：
获取用户为目标对象选择的贴图图像；
基于区域图像在图像帧的原始位置点，确定贴图图像在图像帧中的贴图位置；
将贴图图像按照贴图位置渲染至目标渲染图像中，获得目标贴图图像；
将图像帧对应的目标贴图图像进行视频合成，获得待处理视频对应的贴图目标视频。
根据本公开的一个或多个实施例，对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除区域图像之外的背景图像，包括：
从待处理视频中提取图像帧；
识别图像帧中目标对象的对象轮廓区域；
利用对象轮廓区域分割图像帧，获得对象轮廓区域对应的区域图像和除区域图像之外的背景图像。
根据本公开的一个或多个实施例，还包括：
基于第一用户触发的目标视频采集请求，获取待处理视频；
将图像帧对应的目标渲染图像进行视频合成，获得待处理视频对应的目标视频，之后，还包括：
为第一用户输出目标视频。
根据本公开的一个或多个实施例，为第一用户输出目标视频之后，还包括：
检测第一用户针对目标视频触发的发布请求；
响应于发布请求，将目标视频发布至第一用户的用户视频账户，目标视频供第二用户的第二用户设备读取并播放。
根据本公开的一个或多个实施例，基于第一用户触发的目标视频采集请求，获取待处理视频包括：
接收第一用户设备发送的待处理视频；待处理视频是第一用户的用户设备在用户触发目标视频采集请求时采集获得的；
为第一用户输出目标视频，包括：
发送目标视频至第一用户设备，并控制第一用户设备播放目标视频。
第二方面，根据本公开的一个或多个实施例，提供了一种用于视频合成装置，包括：
图像分割单元，用于对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除区域图像之外的背景图像；
镜像处理单元，用于确定区域图像对应的镜像图像；
第一扩展单元，用于在第一扩展方向对区域图像进行扩展处理，获得至少一个第一扩展图像；至少一个第一扩展图像包括区域图像；
第二扩展单元，用于在第二扩展方向对镜像图像进行扩展处理，获得至少一个第二扩展图像；第一扩展方向和第二扩展方向不同。
图像渲染单元，用于对背景图像、至少一个第二扩展图像和至少一个第一扩展图像进行渲染，获得渲染结束时的目标渲染图像；
视频合成单元，用于将图像帧对应的目标渲染图像进行视频合成，获得待处理视频对应 的目标视频。
第三方面，根据本公开的一个或多个实施例，提供了一种电子设备，包括：至少一个处理器和存储器；
存储器存储计算机执行指令；
至少一个处理器执行存储器存储的计算机执行指令，使得至少一个处理器执行如上第一方面以及第一方面各种可能的设计的视频合成方法。
第四方面，根据本公开的一个或多个实施例，提供了一种计算机可读存储介质，计算机可读存储介质中存储有计算机执行指令，当处理器执行计算机执行指令时，实现如上第一方面以及第一方面各种可能的设计的视频合成方法。
第五方面，根据本公开的一个或多个实施例，提供了一种计算机程序产品，包括计算机程序，计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能的设计的视频合成方法。
第六方面，本公开实施例还提供一种计算机程序，计算机程序被处理器运行时实现如上第一方面以及第一方面各种可能的设计的视频合成方法。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (18)

  1. 一种视频合成方法，包括：
    对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除所述区域图像之外的背景图像；
    确定所述区域图像对应的镜像图像；
    在第一扩展方向对所述区域图像进行扩展处理，获得至少一个第一扩展图像；至少一个所述第一扩展图像包括所述区域图像；
    在第二扩展方向对所述镜像图像进行扩展处理，获得至少一个第二扩展图像；所述第一扩展方向和所述第二扩展方向不同；
    对所述背景图像、至少一个所述第二扩展图像和至少一个所述第一扩展图像进行渲染，获得渲染结束时的目标渲染图像；
    将所述图像帧对应的目标渲染图像进行视频合成，获得所述待处理视频对应的目标视频。
  2. 根据权利要求1所述的方法，其中，所述在第一扩展方向对所述区域图像进行扩展处理，获得至少一个第一扩展图像，包括：
    确定所述目标对象在所述图像帧对应的位移速度；
    以所述区域图像的原始位置点作为基础，按照所述位移速度和第一扩展方向进行位移处理，获得包含所述原始位置点的至少一个正向位移点；
    将所述区域图像复制到所述正向位移点，获得所述区域图像在至少一个所述正向位移点分别对应的第一扩展图像。
  3. 根据权利要求2所述的方法，其中，所述以所述区域图像的原始位置点作为基础，按照所述位移速度和第一扩展方向进行位移处理，获得包含所述原始位置点的至少一个正向位移点，包括：
    确定所述区域图像的区域宽度和所述图像帧的图像宽度；
    基于所述区域宽度和所述图像宽度，结合所述位移速度，在所述第一扩展方向上计算所述区域图像的正向位移数量以及正向偏移步长；
    根据所述区域图像的原始位置，按照正向偏移步长和所述正向位移数量，确定至少一个正向位置点，获得所述原始位置点和至少一个所述正向位置点构成的所述至少一个正向位移点。
  4. 根据权利要求1所述的方法，其中，所述在第二扩展方向对所述镜像图像进行扩展处理，获得至少一个第二扩展图像，包括：
    确定所述目标对象在所述图像帧对应的位移速度；
    以所述区域图像的原始位置点作为基础，基于所述位移速度和第二扩展方向进行位移处理，获得与所述原始位置点相对应的至少一个反向位移点；
    将所述镜像图像复制到所述反向位移点，获得所述镜像图像在至少一个所述反向位移点分别对应的第二扩展图像。
  5. 根据权利要求4所述的方法，其中，所述以所述区域图像的原始位置点作为基础，基于所述位移速度和第二扩展方向进行位移处理，获得与所述原始位置点相对应的至少一个反向位移点，包括：
    确定所述区域图像的区域宽度和所述图像帧的图像宽度；
    基于所述区域宽度和所述图像宽度，结合所述位移速度，在所述第二扩展方向上计算所述区域图像的反向位移数量以及反向偏移步长；
    根据所述区域图像的原始位置，按照反向偏移步长和所述反向偏移数量，确定至少一个反向位置点，获得至少一个所述反向位置点构成的至少一所述反向位移点。
  6. 根据权利要求2或3所述的方法，其中，所述确定所述目标对象在所述图像帧对应的位移速度，包括：
    确定所述图像帧在所述待处理视频对应的前一个图像帧；
    确定所述目标对象在所述前一个图像帧的第一位置和在所述图像帧的第二位置；
    根据所述第一位置和所述第二位置，计算所述目标对象对应的位移量；
    利用所述待处理视频的图像帧显示时间间隔，和所述位移量，计算获得所述目标对象在所述图像帧对应的位移速度。
  7. 根据权利要求1-6任一项所述的方法，其中，所述对所述背景图像、至少一个所述第二扩展图像和至少一个所述第一扩展图像进行渲染，获得渲染结束时的目标渲染图像，包括：
    按照所述背景图像为第一渲染顺序，至少一个所述第二扩展图像为第二渲染顺序和至少一个所述第一扩展图像为第三渲染顺序，依次叠加渲染所述背景图像、至少一个所述第二扩展图像和至少一个所述第一扩展图像，获得渲染结束时的所述目标渲染图像。
  8. 根据权利要求7所述的方法，其中，所述依次叠加渲染所述背景图像、至少一个所述第二扩展图像和至少一个所述第一扩展图像，获得渲染结束时的所述目标渲染图像，包括：
    将所述背景图像在最底层渲染；
    确定至少一个所述第二扩展图像与所述背景图像的中心点分别对应的反向中心距离；
    按照反向中心距离从大到小的顺序，依次渲染至少一个所述第二扩展图像；
    确定至少一个所述第一扩展图像与所述背景图像的中心点分别对应的正向中心距离；
    按照正向中心距离从大到小的顺序，依次渲染至少一个所述第一扩展图像，直至最后一个第一扩展图像渲染结束，获得所述目标渲染图像。
  9. 根据权利要求1至6任一项所述的方法，其中，所述对所述背景图像、至少一个所述第二扩展图像和至少一个所述第一扩展图像进行渲染，获得渲染结束时的目标渲染图像之后，方法还包括：
    获取用户为所述目标对象选择的贴图图像；
    基于所述区域图像在所述图像帧的原始位置点，确定所述贴图图像在所述图像帧中的贴图位置；
    将所述贴图图像按照所述贴图位置渲染至所述目标渲染图像中，获得目标贴图图像；
    将所述图像帧对应的目标贴图图像进行视频合成，获得所述待处理视频对应的贴图目标视频。
  10. 根据权利要求1至9任一项所述的方法，其中，所述对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除所述区域图像之外的背景图像，包括：
    从所述待处理视频中提取图像帧；
    识别所述图像帧中目标对象的对象轮廓区域；
    利用所述对象轮廓区域分割所述图像帧，获得所述对象轮廓区域对应的区域图像和除所述区域图像之外的背景图像。
  11. 根据权利要求1至10任一项所述的方法，其中，所述方法还包括：
    基于第一用户触发的目标视频采集请求，获取所述待处理视频；
    所述将所述图像帧对应的目标渲染图像进行视频合成，获得所述待处理视频对应的目标视频，之后，还包括：
    为所述第一用户输出所述目标视频。
  12. 根据权利要求11所述的方法，其中，所述为所述第一用户输出所述目标视频之后，还包括：
    检测所述第一用户针对所述目标视频触发的发布请求；
    响应于所述发布请求，将所述目标视频发布至所述第一用户的用户视频账户，所述目标视频供第二用户的第二用户设备读取并播放。
  13. 根据权利要求11所述的方法，其中，所述基于第一用户触发的目标视频采集请求，获取所述待处理视频包括：
    接收所述第一用户设备发送的所述待处理视频；所述待处理视频是所述第一用户的用户设备在所述用户触发目标视频采集请求时采集获得的；
    所述为所述第一用户输出所述目标视频，包括：
    发送所述目标视频至第一用户设备，并控制所述第一用户设备播放所述目标视频。
  14. 一种用于视频合成装置，其特征在于，包括：
    图像分割单元，用于对待处理视频中的图像帧进行图像分割，获得目标对象对应的区域图像和除所述区域图像之外的背景图像；
    镜像处理单元，用于确定所述区域图像对应的镜像图像；
    第一扩展单元，用于在第一扩展方向对所述区域图像进行扩展处理，获得至少一个第一扩展图像；至少一个所述第一扩展图像包括所述区域图像；
    第二扩展单元，用于在第二扩展方向对所述镜像图像进行扩展处理，获得至少一个第二扩展图像；所述第一扩展图像和所述第二扩展图像不同；
    图像渲染单元，用于对所述背景图像、至少一个所述第二扩展图像和至少一个所述第一扩展图像进行渲染，获得渲染结束时的目标渲染图像；
    视频合成单元，用于将所述图像帧对应的目标渲染图像进行视频合成，获得所述待处理视频对应的目标视频。
  15. 一种电子设备，其特征在于，包括：处理器、存储器；
    所述存储器存储计算机执行指令；
    所述处理器执行所述存储器存储的计算机执行指令，使得所述处理器配置有如权利要求1至13任一项所述的视频合成方法。
  16. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如权利要求1至13任一项所述的视频合成方法。
  17. 一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现如权利要求1至13任一项所述的视频合成方法。
  18. 一种计算机程序，所述计算机程序被处理器执行时实现如权利要求1至13任一项所述的视频合成方法。
PCT/CN2023/085796 2022-04-29 2023-03-31 视频合成方法、装置、设备、介质及产品 WO2023207522A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210476242.2A CN114881901A (zh) 2022-04-29 2022-04-29 视频合成方法、装置、设备、介质及产品
CN202210476242.2 2022-04-29

Publications (1)

Publication Number Publication Date
WO2023207522A1 true WO2023207522A1 (zh) 2023-11-02

Family

ID=82674312

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/085796 WO2023207522A1 (zh) 2022-04-29 2023-03-31 视频合成方法、装置、设备、介质及产品

Country Status (2)

Country Link
CN (1) CN114881901A (zh)
WO (1) WO2023207522A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114881901A (zh) * 2022-04-29 2022-08-09 北京字跳网络技术有限公司 视频合成方法、装置、设备、介质及产品
CN117853324A (zh) * 2022-09-29 2024-04-09 北京字跳网络技术有限公司 扩展图像的方法、装置及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110070551A (zh) * 2019-04-29 2019-07-30 北京字节跳动网络技术有限公司 视频图像的渲染方法、装置和电子设备
CN111586319A (zh) * 2020-05-27 2020-08-25 北京百度网讯科技有限公司 视频的处理方法和装置
CN112235604A (zh) * 2020-10-20 2021-01-15 广州博冠信息科技有限公司 渲染方法及装置、计算机可读存储介质、电子设备
CN112929582A (zh) * 2021-02-04 2021-06-08 北京字跳网络技术有限公司 一种特效展示方法、装置、设备及介质
CN114881901A (zh) * 2022-04-29 2022-08-09 北京字跳网络技术有限公司 视频合成方法、装置、设备、介质及产品

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110070551A (zh) * 2019-04-29 2019-07-30 北京字节跳动网络技术有限公司 视频图像的渲染方法、装置和电子设备
CN111586319A (zh) * 2020-05-27 2020-08-25 北京百度网讯科技有限公司 视频的处理方法和装置
CN112235604A (zh) * 2020-10-20 2021-01-15 广州博冠信息科技有限公司 渲染方法及装置、计算机可读存储介质、电子设备
CN112929582A (zh) * 2021-02-04 2021-06-08 北京字跳网络技术有限公司 一种特效展示方法、装置、设备及介质
CN114881901A (zh) * 2022-04-29 2022-08-09 北京字跳网络技术有限公司 视频合成方法、装置、设备、介质及产品

Also Published As

Publication number Publication date
CN114881901A (zh) 2022-08-09

Similar Documents

Publication Publication Date Title
WO2023207522A1 (zh) 视频合成方法、装置、设备、介质及产品
WO2019242262A1 (zh) 基于增强现实的远程指导方法、装置、终端和存储介质
US11557083B2 (en) Photography-based 3D modeling system and method, and automatic 3D modeling apparatus and method
CN111292420B (zh) 用于构建地图的方法和装置
WO2020248900A1 (zh) 全景视频的处理方法、装置及存储介质
EP4050305A1 (en) Visual positioning method and device
WO2022007565A1 (zh) 增强现实的图像处理方法、装置、电子设备及存储介质
WO2023273036A1 (zh) 导航方法、装置、电子设备及可读存储介质
WO2024051536A1 (zh) 直播特效渲染方法、装置、设备、可读存储介质及产品
WO2024104248A1 (zh) 虚拟全景图的渲染方法、装置、设备及存储介质
WO2019007372A1 (zh) 模型显示方法和装置
US11494961B2 (en) Sticker generating method and apparatus, and medium and electronic device
WO2022247630A1 (zh) 图像处理方法、装置、电子设备及存储介质
KR20230130748A (ko) 이미지 처리 방법 및 장치, 디바이스 및 매체
WO2022237048A1 (zh) 位姿获取方法、装置、电子设备、存储介质及程序
JP7387001B2 (ja) 画像合成方法、装置、およびストレージ媒体
WO2024051756A1 (zh) 特效图像绘制方法、装置、设备及介质
CN109816791B (zh) 用于生成信息的方法和装置
US10341683B1 (en) Apparatus and method to reduce an amount of coordinate data representing an object taken by an imaging device in a three dimensional space
CN115578432B (zh) 图像处理方法、装置、电子设备及存储介质
WO2023088127A1 (zh) 室内导航方法、服务器、装置和终端
WO2023172195A2 (zh) 线条特效处理方法、装置、电子设备、存储介质及产品
CN116824688A (zh) 小腿动作捕捉方法、系统及存储介质
WO2019114653A1 (zh) 一种导航引导图的生成方法和装置
CN112037227A (zh) 视频拍摄方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23794957

Country of ref document: EP

Kind code of ref document: A1