CN113810725A - Video processing method, device, storage medium and video communication terminal - Google Patents
Video processing method, device, storage medium and video communication terminal Download PDFInfo
- Publication number
- CN113810725A CN113810725A CN202111187261.5A CN202111187261A CN113810725A CN 113810725 A CN113810725 A CN 113810725A CN 202111187261 A CN202111187261 A CN 202111187261A CN 113810725 A CN113810725 A CN 113810725A
- Authority
- CN
- China
- Prior art keywords
- video
- target object
- frame rate
- scene
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 28
- 238000004891 communication Methods 0.000 title claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 65
- 238000000034 method Methods 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 9
- 230000000694 effects Effects 0.000 abstract description 36
- 230000005540 biological transmission Effects 0.000 abstract description 30
- 238000000605 extraction Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/631—Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
Abstract
The application discloses a video processing method, a video processing device, a storage medium and a video communication terminal, wherein the video processing device comprises: acquiring a first video of a target object area in a shooting scene; acquiring a second video of a non-target object area in a shooting scene, wherein the video frame rate of the first video is greater than that of the second video; and transmitting the first video and the second video to a receiving terminal. By obtaining the second video with the video frame rate lower than that of the first video, the overall data volume of the video is reduced, the transmission efficiency is improved, the progress of receiving the first video and the second video by the receiving terminal is ensured, the situation that the first video and the second video are played on the receiving terminal in a blocked mode is avoided, and the video display effect is improved.
Description
Technical Field
The present application relates to the field of video communication technologies, and in particular, to a video processing method, an apparatus, a storage medium, and a video communication terminal.
Background
According to the requirements of a video conference, terminal equipment such as a conference machine is required to be used, video acquisition and transmission are carried out at a sending terminal, transmitted video is received and displayed at a receiving terminal, and then video communication between the two terminals is achieved.
The resolution of the video acquired by the sending terminal is too high or too low, which may affect the display effect of the video at the receiving terminal, for example, the video image displayed by the receiving terminal is blurred, noisy or mosaic appears, or a display stuck phenomenon exists.
Disclosure of Invention
The embodiment of the application provides a video processing method, a video processing device, a storage medium and a video communication terminal, which can improve the video display effect.
In a first aspect, an embodiment of the present application provides a video processing method, including:
acquiring a first video of a target object area in a shooting scene;
acquiring a second video of a non-target object area in a shooting scene, wherein the video frame rate of the first video is greater than that of the second video;
and transmitting the first video and the second video to a receiving terminal.
In a second aspect, an embodiment of the present application further provides a video processing apparatus, including:
the first video processing module is used for acquiring a first video of a target object area in a shooting scene;
the second video processing module is used for acquiring a second video of a non-target object area in a shooting scene, and the video frame rate of the first video is greater than that of the second video;
and the video sending module is used for sending the first video and the second video to the receiving terminal.
In a third aspect, embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute a video processing method as provided in any of the embodiments of the present application.
In a fourth aspect, an embodiment of the present application further provides a video communication terminal, including a processor and a memory, where the memory has a computer program, and the processor is configured to execute the video processing method according to any embodiment of the present application by calling the computer program.
According to the technical scheme provided by the embodiment of the application, the first video of the target object area in the shooting scene and the second video of the non-target object area in the shooting scene are obtained at the sending terminal, so that the target object area and the non-target object area in the same shooting scene are separated, the first video of the target object area and the second video of the non-target object area are obtained, and the video frame rate of the first video is greater than that of the second video. Compared with the prior art that a target object area and a non-target object area are not distinguished, and a video is transmitted to a receiving terminal, the processing mode in the prior art is prone to causing video blocking due to overlarge video data amount, and further influences the display effect of the video on the receiving terminal. In the embodiment of the application, because the video frame rate of the first video is higher and the video frame rate of the second video is lower, the overall data volume of the video can be reduced, and further the transmission efficiency of the first video and the second video is improved, so that the first video and the second video on the receiving terminal can be smoothly displayed after the first video is sent to the receiving terminal, and the display effect of the video is improved. Moreover, the first video and the second video can be transmitted respectively, the transmission pressure can be reduced, the video transmission efficiency is improved, the smoothness of the transmission process is further guaranteed, smooth display can be achieved after the receiving terminal receives the first video and the second video, and the video display effect is guaranteed. Therefore, the method provided by the embodiment of the application can not cause video playing to be blocked when the video is displayed, the display effect of the target object area is clear, and mosaic or noise phenomenon can not occur.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart illustrating a method for a sending terminal to execute a video processing according to an embodiment of the present application.
Fig. 2 is a flowchart illustrating a method for a receiving terminal to execute a video processing according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present application.
Fig. 4 is a schematic structural diagram of a video communication terminal according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without inventive step, are within the scope of the present application.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
An execution main body of the video processing method can be the video processing device provided by the embodiment of the application or a video communication terminal integrated with the video processing device. The video processing device can be realized in a hardware or software mode, and the video communication terminal can be a terminal device supporting video call, such as a smart phone, a tablet computer, a palm computer, a conference machine and the like.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a sending terminal executing a video processing method according to an embodiment of the present application. The specific flow of the video processing method provided by the embodiment of the application can be as follows:
101. a first video of a target object area in a shooting scene is acquired.
Illustratively, a transmitting terminal acquires a first video of a target object area in a shooting scene. The first video of the target object area can be shot through a camera configured on the sending terminal, and the first video can also be sent to the sending terminal after the first video of the target object area is shot through an external camera. It is to be understood that the manner of acquiring the first video is not limited herein as long as the first video can be acquired.
The target object area refers to an area where the target object is located, and when the target object moves, the target object area changes correspondingly. The target object may be a person, a moving object, or the like, and as long as a certain attribute of the target object is set, the target object may be distinguished according to the attribute characteristics to obtain the first video of the target object.
102. And acquiring a second video of a non-target object area in the shooting scene, wherein the video frame rate of the first video is greater than that of the second video.
In the same shooting scene, after the target object area is located, the area except the target object area is a non-target object area, and the non-target object area changes along with the change of the target object area.
For example, the mode of acquiring the second video may be obtained by shooting the non-target object area through a camera configured in the sending terminal, or by shooting the non-target object area through another external camera.
The first video and the second video can be obtained by shooting with the same camera or different cameras, and the method is not limited herein as long as two videos can be obtained, wherein one video is the first video and the other video is the second video.
The video frame rate of the first video is greater than that of the second video, and as the target object is an object needing attention, the required display effect can be achieved as long as the first video is ensured to have a higher video frame rate, and the video frame rate of the second video is lower than that of the first video, wherein the video frame rate of the second video can also be set to be a minimum value, and when the first video is acquired subsequently, the acquired video frame rate of the first video is necessarily greater than that of the second video, so that the overall data volume of the first video and the second video is reduced, and the transmission efficiency is ensured. It is to be understood that the specific setting manner of the video frame rates of the first video and the second video is not limited herein, and the protection scope of the present application is only required to implement a manner that the video frame rate of the acquired first video is greater than the video frame rate of the second video.
By obtaining the first video with a higher video frame rate and the second video with a lower video frame rate, the overall data volume of the video can be reduced due to the higher video frame rate of the first video and the lower video frame rate of the second video, and the transmission efficiency of the first video and the second video is further improved, so that the first video and the second video can be smoothly displayed on the receiving terminal after being sent to the receiving terminal, the display effect of the video is ensured, video playing is not blocked, the display effect of a target object area is clear, and mosaic or noise phenomenon is not caused.
103. And transmitting the first video and the second video to a receiving terminal.
The first video and the second video may be sent to the receiving terminal as two video streams, respectively, or the first video and the second video may be combined into one video stream and sent to the receiving terminal, and the data terminal analyzes the video stream.
Illustratively, the first video and the second video can be compressed and then transmitted to the receiving terminal, so that the transmission amount of the video is reduced, the transmission efficiency is improved, and accordingly, the display effect of the video on the receiving terminal is ensured.
In particular implementation, the present application is not limited by the execution sequence of the described steps, and some steps may be performed in other sequences or simultaneously without conflict.
The method and the device have the advantages that the first video with the high frame rate and the second video with the low frame rate are obtained through the sending terminal in the implementation of the method and the device, and the first video and the second video are sent to the receiving terminal. Simultaneously, because the video frame rate of first video is higher, do not influence the broadcast of first video, and first video corresponds and shows the target object, can guarantee the display effect of target object on receiving terminal, consequently, from whole display effect, this application scheme can reach and guarantee not blocking at receiving terminal broadcast video, also can guarantee that the target object has sufficient definition for the display effect of target object promotes.
It can understand, when first video and second video in this application embodiment are high definition video, through the scheme of this application embodiment, can reduce the data size in first video and the second video transmission process equally, and then guarantee that there is not the broadcast card pause phenomenon when receiving terminal receives first video and second video, and then guarantee the broadcast smoothness degree of first video and second video, moreover, first video recording is the video of target object, through carrying out smooth broadcast to first video at receiving terminal, also can guarantee the definition of first video display, and improve the broadcast smoothness degree, and then improve holistic video display effect.
Based on this, the scheme that this application embodiment provided not only can guarantee the broadcast smoothness degree of video at receiving terminal, also can guarantee the definition that shows at receiving terminal to it has been solved among the prior art to play high definition video and has had the video card pause, and play non-high definition video and have phenomena such as video blurring, can realize guaranteeing high definition to show simultaneously and broadcast the smoothness degree, has greatly improved the video display effect, does benefit to more and uses and popularize in the video communication field.
The method according to the preceding embodiment is illustrated in further detail below by way of example.
The above-mentioned modes of acquiring the first video of the target object region in the shooting scene and acquiring the second video of the non-target object region in the shooting scene are various:
for example, a first video with a high video frame rate can be obtained by directly setting the shooting frame rate of the camera, and a second video with a low video frame rate can be obtained by shooting.
For another example, it is not necessary to set the shooting frame rate of the camera in advance, and after the first video and the second video are obtained by shooting, the video frame rate of the first video is subjected to high frame rate coding processing, and the video frame rate of the second video is subjected to low frame rate coding processing, so that the video frame rate of the first video is greater than the video frame rate of the second video.
It is to be understood that the setting of the video frame rate of the first video and the setting of the video frame rate of the second video may be implemented by setting the shooting frame rate of the camera, performing frame rate coding after the shooting, or combining the setting of the camera and the frame rate coding, and the specific implementation manner is not limited herein as long as the first video with a higher video frame rate relative to the second video can be implemented.
It should be noted that, in the embodiment of the present application, the first video with a higher video frame rate, the second video with a lower video frame rate, the video frame rate of the first video being higher/higher than the video frame rate of the second video, and the high frame rate coding and the low frame rate coding, where the description about high and low is only a relative expression, and only refers to a relative magnitude of the video frame rate.
As for the manner of acquiring the first video and the second video, in the following embodiments, two manners of the first video and the second video will be described by way of example, and it should be understood that the two manners of acquiring the first video and the second video should not be construed as limiting the present application.
In an embodiment, the first video and the second video may be acquired by setting a camera, where step 101 includes:
shooting a target object area in a shooting scene based on a first preset frame rate to obtain a first video;
step 102 comprises:
and shooting a non-target object area in the shooting scene based on a second preset frame rate to obtain a second video, wherein the first preset frame rate is greater than the second preset frame rate.
For example, the target object area and the non-target object area may be sequentially photographed by one camera, or the target object area and the non-target object area may be photographed by two cameras respectively.
When shooting through one camera, the shooting frame rate of the camera can be set to a first preset frame rate, after a first video of a target object is obtained through shooting, the shooting frame rate of the camera is set to a second preset frame rate, and a second video of a non-target object is obtained through shooting. By performing shooting in this way, the number of cameras can be reduced, and the specific positions of the target object and the non-target object can be distinguished.
Of course, the two cameras may also be used to respectively capture images, wherein the frame rate of one camera may be set to a first preset frame rate, so as to obtain a first video with the first preset frame rate, and the frame rate of the other camera may be set to a second preset frame rate, so as to obtain a second video with the second preset frame rate.
In some embodiments, the AI camera may also be used to automatically capture and track facial images to distinguish between target object regions and non-target object regions in the scene.
It is to be understood that the specific values of the first preset frame rate and the second preset frame rate are not limited herein, and the values can be set by those skilled in the art according to actual requirements.
In this embodiment, by setting the shooting frame rate of the camera, the first video with the first preset frame rate and the second video with the second preset frame rate can be directly obtained, and the first preset frame rate is greater than the second preset frame rate, so that the video frame rate is prevented from being subsequently processed, and the video processing efficiency is improved.
In another embodiment, the video coding processing may be further performed on the acquired first video and the acquired second video, so that the video frame rate of the first video is a first preset frame rate, and the video frame rate of the second video is a second preset frame rate.
Wherein step 101 comprises:
shooting a shooting scene to obtain a scene image of the shooting scene;
and according to a first preset frame rate, carrying out video coding on the image content of the target object region in the scene image to obtain a first video.
Step 102 comprises:
and according to a second preset frame rate, carrying out video coding on the content of the non-target object region in the scene image to obtain a second video, wherein the first preset frame rate is greater than the second preset frame rate.
The method comprises the steps of shooting a scene image through the same frame rate, when frame rate processing is carried out, obtaining a video of a target object region and a video of a non-target object region by dividing the target object region and the non-target object region from the scene image, carrying out video coding processing on the video of the target object region according to a first preset frame rate to obtain a first video, carrying out video coding processing on the video of the non-target object region according to a second preset frame rate to obtain a second video, wherein the first preset frame rate is greater than the second preset frame rate, and therefore the video frame rate of the first video is greater than the second preset video frame rate.
For example, the segmentation of the target object region and the non-target object region may be obtained by using an image segmentation model, or may be obtained by performing feature recognition on an image to further extract image content of a region where the target object is located.
In the embodiment, the video coding is performed on the image content of the target object region at the first preset frame rate to obtain the first video, and the video coding is performed on the image content of the non-target object region at the second preset frame rate to obtain the second video, so that the video frame rates of the first video and/or the second video can be freely adjusted, the real-time adjustment on the video frame rates is realized, the data transmission requirements are better met, and the limitation that the camera can only set a limited number of shooting frame rates is solved.
Based on this, the first preset frame rate and the second preset frame rate in this embodiment may not be fixed values, and may be automatically adjusted according to actual requirements, or may be set and adjusted by a user, and when the first preset frame rate and the second preset frame rate are automatically adjusted, the adjustment may be based on network transmission amount or memory occupation amount, and a specific implementation manner is not described in detail herein, as long as the first preset frame rate and/or the second preset frame rate can be adaptively adjusted.
In some embodiments, after the capturing the captured scene and obtaining the scene image of the captured scene, the method further includes:
blurring the image content of the non-target object region in the scene image to obtain a blurred scene image;
and carrying out video coding on the blurred scene image according to a first preset frame rate to obtain a first video.
For example, after the scene image is obtained by shooting, blurring processing may be performed on the image content in the scene image where the non-target object region is located, so that the image of the non-target object is blurred to form contrast with the target object, so as to highlight the display effect of the target object, and make the target object displayed more clearly visually.
In this embodiment, it is not necessary to encode the image of the non-target object region according to the second preset frame rate, and the bit depth of each pixel of the image can be changed by blurring the image content of the non-target object region, so that the data amount of the image of the non-target object region is reduced, the transmission efficiency can also be improved, and the display effect of the target object at the receiving terminal is further improved.
For sending the first video and the second video to the receiving terminal for playing, the first video and the second video may be processed by the sending terminal and then sent to the receiving terminal for direct playing and displaying, or the first video and the second video may be processed and then played on the receiving terminal after being sent to the receiving terminal. Therefore, the processing of the first video and the second video is not limited to being based on the transmitting terminal or the receiving terminal.
There are also a number of ways to process the first video and the second video:
for example, the frame interpolation processing may be performed on the second video to obtain a video frame rate that is the same as that of the first video, and then each frame of the first video and the second video is spliced to obtain a spliced image and played frame by frame.
For another example, the first video and the second video may be played simultaneously, and the first video may be played overlaid on the second video as long as the display of the first video is not affected.
For another example, the third video may be formed by performing frame extraction processing on the second video, and further extracting the image frames with the timestamps consistent with the image frames of the first video from the second video.
In the following embodiments, only two ways of frame interpolation and frame extraction are illustrated, but it should be noted that the two illustrations should not be construed as limiting the application.
In an embodiment, the second video is processed in a frame-extracting manner, and step 103 includes:
extracting image frames matched with the first video from the second video according to the time stamps of the image frames in the first video and the second video to obtain a third video;
and transmitting the first video and the third video to a receiving terminal.
For example, the image frames in the second video may be extracted as time stamps of the image frames in the first video, and the extracted image frames may be used as a third video, where each image frame in the third video has a matching image frame in the first video.
The consistency with the time stamp-based method refers to that the image frames synchronized with the image frames of the first video in time, namely the image frames coincident with the time, are screened out from the image frames of the second video, and the image frames that do not coincide with the time can be directly discarded, so that the frame extraction processing of the second video is realized.
After the third video is obtained, each image frame in the third video has the image frame matched with the timestamp in the first video, and in this way, the image frame in the second video can be smaller, so that the data volume of the obtained third video is smaller, and the transmission efficiency of the first video and the third video is greatly improved.
In another embodiment, the second video is processed in a frame interpolation manner, and step 103 includes:
copying image frames of the second video according to the first preset frame rate, and inserting the copied image frames into the second video frame by frame to obtain a third video;
and transmitting the first video and the third video to a receiving terminal.
Illustratively, the copy rate of the image frames of the second video may be obtained by calculating a difference between a first preset frame rate and a second preset frame rate, and after the image frames of the second video are copied, the image frames of the second video are inserted between two temporally adjacent image frames, so that the second video is subjected to frame insertion processing, the video frame rate of the second video is increased, and a third video is obtained, wherein the video frame rate of the third video is the same as the video frame rate of the first video.
The frame interpolation mode can realize synchronous playing of the first video and the third video, and reduces the processing process of the first video and the third video. The simple superposition of the first video and the third video can be realized, the video of the shooting scene can be restored, and the video display effect of the shooting scene can be restored.
In some embodiments, the shooting scene is a conference scene, the target object region is a region where a speaker is located in the conference scene, and the non-target object region is a region other than the speaker, wherein the non-target object region may be a conference background;
acquiring a first video of a target object area in a shooting scene, comprising:
shooting a conference scene to obtain a conference image of the conference scene;
acquiring a first video of a speaker;
acquiring a second video of a non-target object area in a shooting scene, wherein the second video comprises:
and matching the conference images from a preset conference database to obtain a second video.
Illustratively, only the speaker in the conference scene may be photographed and video encoding processing may be performed, thereby obtaining the first video. And for a second video of a non-target object area, the second video can be obtained by matching the historical data.
The preset conference database stores videos of different conference scenes, after the processing mode of the embodiment is adopted, the obtained second videos can be stored in the preset conference database, when the second videos are needed to be used as backgrounds, the videos corresponding to conference images can be directly matched from the preset conference database, and when the videos are not matched, the second videos can be obtained through shooting.
It can be understood that the target object may be a speaker in the conference or an object moving in the conference, and since images except for the target object may be referred to as a conference background and the conference background is static, only the first video of the speaker in the conference may be captured, and the video may not be captured for the conference background as long as it is ensured that the first video is captured and sent to the receiving terminal.
Moreover, because the conference background is static, the frame rate of the second video of the conference background is reduced, and the second video is not greatly changed when being played, so that the display effect is not influenced. Of course, a pure color background picture may also be set instead of the second video, so as to greatly reduce the amount of video data and improve the video display effect.
According to the embodiment of the application, the conference scene is shot, and then the conference background can be matched according to the shot conference image, wherein the conference background can be a video or a picture, the conference background is prevented from being shot in such a way, or the conference image is processed to obtain the video of the conference background, so that the process of obtaining a second video is omitted, the video processing efficiency is greatly improved, and the video display effect can be greatly improved on a receiving terminal.
As can be seen from the above, the receiving terminal according to the embodiment of the present application may also be used as a sending terminal to implement the above-described video processing method, and when receiving a video sent from the sending terminal as the receiving terminal, the receiving terminal may process and play the received two videos, or may directly play the processed two videos. In the following embodiments, a manner of processing and playing two received videos will be described in detail.
As shown in fig. 2, fig. 2 is a schematic flowchart of a method for a receiving terminal to execute video processing according to an embodiment of the present application, where in this embodiment, after step 103, the method further includes:
201. receiving a fourth video and a fifth video, wherein the video frame rate of the fourth video is greater than that of the fifth video;
202. splicing the image frames in the fifth video to the image frames in the fourth video according to the time stamps to obtain a spliced sixth video;
203. and playing the sixth video.
If the fifth video is a video subjected to frame extraction processing at the sending terminal, each image frame of the fifth video has a matched image frame in the fourth video, and the image frames of the fifth video and the fourth video can be spliced directly according to the timestamp, where it should be noted that the splicing here refers to alignment processing of a target object region in the fourth video and a non-target object region in the fifth video, so that the spliced sixth video can achieve an effect of restoring an image of a shot scene in a visual effect.
If the fifth video is not subjected to frame extraction processing at the sending terminal, the fourth video and the fifth video can be directly and differently mixed and played, and only the target object area and the non-target object area need to be aligned in the image frame.
Naturally, frame interpolation processing may also be performed on the fifth video at the receiving terminal, and the frame interpolation processing mode may refer to the content mentioned in the above embodiment, which is not described herein again, and the fifth video is processed into a video having the same video frame rate as the fourth video, that is, the two videos may be spliced to obtain the sixth video, so that the effect of synchronous playing is achieved.
And processing the fifth video and the fourth video through a neural network restoration model, and inputting the fifth video and the fourth video into the neural network restoration model to obtain a restored sixth video. The neural network restoration model can be obtained through sample set training.
As can be seen from the above, when the video processing method provided in the embodiment of the present invention is used as a sending terminal, a first video with a higher video frame rate and a second video with a lower video frame rate can be obtained, and because the video frame rate of the first video is higher and the video frame rate of the second video is lower, the data size of the whole video can be reduced, and further the transmission efficiency of the first video and the second video can be improved, so that it can be ensured that the first video and the second video on the receiving terminal can be smoothly displayed after the first video is sent to the receiving terminal, the display effect of the video is ensured, the video playing jam cannot be caused, the display effect of the target object region is clear, and the mosaic or noise phenomenon cannot occur. Moreover, still through taking out frame processing to the second video, also can greatly reduce the data bulk, improve transmission efficiency to and through dividing into first video and second video with the video and transmitting respectively, also alleviate transmission pressure, improve transmission efficiency, based on this, the scheme of this application embodiment is through improving transmission efficiency, and then guarantees the video at receiving terminal's display effect, moreover, to high definition video, this scheme is suitable for equally to this has solved the poor problem of video display effect among the prior art.
In one embodiment, a video processing apparatus is also provided. Referring to fig. 3, fig. 3 is a schematic structural diagram of a video processing apparatus 300 according to an embodiment of the present disclosure. The video processing apparatus 300 is applied to a video communication terminal, and the video processing apparatus 300 includes a first video processing module 301, a second video processing module 302, and a video sending module 303, as follows:
the first video processing module 301 is configured to acquire a first video of a target object area in a shooting scene;
the second video processing module 302 is configured to obtain a second video of a non-target object region in a shooting scene, where a video frame rate of the first video is greater than a video frame rate of the second video;
a video sending module 303, configured to send the first video and the second video to the receiving terminal.
In some embodiments, the first video processing module 301 is further configured to:
extracting image frames matched with the first video from the second video according to the time stamps of the image frames in the first video and the second video to obtain a third video;
and transmitting the first video and the third video to a receiving terminal. In some embodiments, the first video processing module 301 is further configured to:
shooting a shooting scene to obtain a scene image of the shooting scene;
and according to a first preset frame rate, carrying out video coding on the image content of the target object region in the scene image to obtain a first video.
In some embodiments, the first video processing module 301 is further configured to:
blurring the image content of the non-target object region in the scene image to obtain a blurred scene image;
and carrying out video coding on the blurred scene image according to a first preset frame rate to obtain a first video.
In some embodiments, the second video processing module 302 is further configured to:
and according to a second preset frame rate, carrying out video coding on the content of the non-target object region in the scene image to obtain a second video, wherein the first preset frame rate is greater than the second preset frame rate.
In some embodiments, the video sending module 303 is further configured to:
extracting image frames matched with the first video from the second video according to the time stamps of the image frames in the first video and the second video to obtain a third video;
and transmitting the first video and the third video to a receiving terminal.
In some embodiments, the shooting scene is a conference scene, and the target object area is an area where a speaker is located in the conference scene:
the first video processing module 301 is further configured to:
shooting a conference scene to obtain a conference image of the conference scene;
acquiring a first video of a speaker;
the second video processing module 302 is further configured to:
and matching the conference images from a preset conference database to obtain a second video.
In some embodiments, the video processing apparatus further comprises:
a video receiving module 304, configured to receive a fourth video and a fifth video, where a video frame rate of the fourth video is greater than a video frame rate of the fifth video;
splicing the image frames in the fifth video to the image frames in the fourth video according to the time stamps to obtain a spliced sixth video;
and playing the sixth video.
It should be noted that the video processing apparatus 300 provided in this embodiment of the present application and the video processing method in the foregoing embodiment belong to the same concept, and any method provided in the video processing method embodiment can be implemented by the video processing apparatus 300, and the specific implementation process thereof is described in detail in the video processing method embodiment, and is not described herein again.
As can be seen from the above, when the video processing apparatus provided in the embodiment of the present application is used as a sending terminal, a first video with a higher video frame rate and a second video with a lower video frame rate can be obtained through obtaining the first video with the higher video frame rate and obtaining the second video with the lower video frame rate, and therefore, the overall data amount of the video can be reduced, and further, the transmission efficiency of the first video and the second video can be improved, so that it can be ensured that the first video and the second video on the receiving terminal can be smoothly displayed after the first video is sent to the receiving terminal, the display effect of the video is ensured, the video playing jam cannot be caused, the display effect of the target object region is clear, and the mosaic or noise phenomenon cannot occur. Moreover, still through taking out frame processing to the second video, also can greatly reduce the data bulk, improve transmission efficiency to and through dividing into first video and second video with the video and transmitting respectively, also alleviate transmission pressure, improve transmission efficiency, based on this, the scheme of this application embodiment is through improving transmission efficiency, and then guarantees the video at receiving terminal's display effect, moreover, to high definition video, this scheme is suitable for equally to this has solved the poor problem of video display effect among the prior art.
The embodiment of the application also provides a video communication terminal, which can be terminal equipment supporting video call, such as a smart phone, a tablet computer, a palm computer, a conference machine and the like. As shown in fig. 4, fig. 4 is a schematic structural diagram of a video communication terminal according to an embodiment of the present application. The video communication terminal 400 includes a processor 401 having one or more processing cores, a memory 402 having one or more computer-readable storage media, and a computer program stored on the memory 402 and executable on the processor. The processor 401 is electrically connected to the memory 402. It will be understood by those skilled in the art that the video communication terminal configuration shown in the figures is not intended to be limiting of video communication terminals and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
The processor 401 is a control center of the video communication terminal 400, connects various parts of the entire video communication terminal 400 using various interfaces and lines, and performs various functions of the video communication terminal 400 and processes data by running or loading software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby integrally monitoring the video communication terminal 400.
In this embodiment, the processor 401 in the video communication terminal 400 loads instructions corresponding to one or more processes of an application program into the memory 402 according to the following steps, and the processor 401 runs the application program stored in the memory 402, so as to implement various functions:
acquiring a first video of a target object area in a shooting scene;
acquiring a second video of a non-target object area in a shooting scene, wherein the video frame rate of the first video is greater than that of the second video;
and transmitting the first video and the second video to a receiving terminal.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
Therefore, the video communication terminal provided by the embodiment can obtain the first video with a higher video frame rate and obtain the second video with a lower video frame rate by using the video communication terminal as the sending terminal, and can reduce the overall data volume of the video due to the higher video frame rate of the first video and the lower video frame rate of the second video, so as to improve the transmission efficiency of the first video and the second video, thereby ensuring that the first video and the second video can be smoothly displayed on the receiving terminal after the first video is sent to the receiving terminal, ensuring the display effect of the video, and not causing the video to be played and blocked, and the display effect of the target object area is clear, and the mosaic or noise phenomenon can not occur. Moreover, still through taking out frame processing to the second video, also can greatly reduce the data bulk, improve transmission efficiency to and through dividing into first video and second video with the video and transmitting respectively, also alleviate transmission pressure, improve transmission efficiency, based on this, the scheme of this application embodiment is through improving transmission efficiency, and then guarantees the video at receiving terminal's display effect, moreover, to high definition video, this scheme is suitable for equally to this has solved the poor problem of video display effect among the prior art.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, the embodiments of the present application provide a computer-readable storage medium, and it can be understood by those skilled in the art that all or part of the steps in the method for implementing the embodiments described above can be implemented by hardware that is instructed to be implemented by a program, and the program can be stored in a computer-readable storage medium, and when executed, the program includes the following steps:
acquiring a first video of a target object area in a shooting scene;
acquiring a second video of a non-target object area in a shooting scene, wherein the video frame rate of the first video is greater than that of the second video;
and transmitting the first video and the second video to a receiving terminal.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
The storage medium may be ROM/RAM, magnetic disk, optical disk, etc. Since the computer program stored in the storage medium can execute the steps in any video processing method provided in the embodiments of the present application, beneficial effects that can be achieved by any video processing method provided in the embodiments of the present application can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.
The video processing method, the video processing device, the video processing medium and the video communication terminal provided by the embodiments of the present application are described in detail above, and a specific example is applied in the description to explain the principle and the implementation of the present application, and the description of the embodiments is only used to help understanding the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A video processing method, comprising:
acquiring a first video of a target object area in a shooting scene;
acquiring a second video of a non-target object area in the shooting scene, wherein the video frame rate of the first video is greater than that of the second video;
and sending the first video and the second video to a receiving terminal.
2. The video processing method of claim 1, wherein said transmitting the first video and the second video to a receiving terminal comprises:
extracting image frames matched with the first video from the second video according to the time stamps of the image frames in the first video and the second video to obtain a third video;
and sending the first video and the third video to the receiving terminal.
3. The video processing method of claim 1, wherein said obtaining a first video of a target object region in a capture scene comprises:
shooting the shooting scene to obtain a scene image of the shooting scene;
and according to a first preset frame rate, carrying out video coding on the image content of the target object area in the scene image to obtain the first video.
4. The video processing method of claim 3, wherein after the capturing the captured scene to obtain the scene image of the captured scene, further comprising:
blurring the image content of the non-target object area in the scene image to obtain a blurred scene image;
and according to the first preset frame rate, carrying out video coding on the blurred scene image to obtain the first video.
5. The video processing method of claim 3, wherein said obtaining a second video of a non-target object region in the captured scene comprises:
and performing video coding on the content of the non-target object area in the scene image according to a second preset frame rate to obtain a second video, wherein the first preset frame rate is greater than the second preset frame rate.
6. The video processing method according to any one of claims 1 to 5, wherein the shooting scene is a conference scene, and the target object area is an area where a speaker is located in the conference scene;
the acquiring of the first video of the target object area in the shooting scene includes:
shooting the conference scene to obtain a conference image of the conference scene;
acquiring a first video of the speaker;
the acquiring of the second video of the non-target object area in the shooting scene includes:
and matching the conference images from a preset conference database to obtain the second video.
7. The video processing method of any of claims 1-5, wherein the method further comprises:
receiving a fourth video and a fifth video, wherein the video frame rate of the fourth video is greater than that of the fifth video;
splicing the image frames in the fifth video to the image frames of the fourth video according to the time stamps to obtain a spliced sixth video;
and playing the sixth video.
8. A video processing apparatus, comprising:
the first video processing module is used for acquiring a first video of a target object area in a shooting scene;
the second video processing module is used for acquiring a second video of a non-target object area in the shooting scene, and the video frame rate of the first video is greater than that of the second video;
and the video sending module is used for sending the first video and the second video to a receiving terminal.
9. A computer-readable storage medium, on which a computer program is stored, which, when run on a computer, causes the computer to carry out a video processing method according to any one of claims 1 to 7.
10. A video communication terminal comprising a processor and a memory, said memory storing a computer program, wherein said processor is adapted to execute the video processing method according to any of claims 1 to 7 by calling said computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111187261.5A CN113810725A (en) | 2021-10-12 | 2021-10-12 | Video processing method, device, storage medium and video communication terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111187261.5A CN113810725A (en) | 2021-10-12 | 2021-10-12 | Video processing method, device, storage medium and video communication terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113810725A true CN113810725A (en) | 2021-12-17 |
Family
ID=78897545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111187261.5A Pending CN113810725A (en) | 2021-10-12 | 2021-10-12 | Video processing method, device, storage medium and video communication terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113810725A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114520873A (en) * | 2021-12-31 | 2022-05-20 | 广州文远知行科技有限公司 | Sweeper video acceleration method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007221367A (en) * | 2006-02-15 | 2007-08-30 | Mitsubishi Electric Corp | Image transmission apparatus and system |
| CN105635624A (en) * | 2014-10-27 | 2016-06-01 | 华为技术有限公司 | Video image processing method, device and system |
| CN109068052A (en) * | 2018-07-24 | 2018-12-21 | 努比亚技术有限公司 | video capture method, mobile terminal and computer readable storage medium |
| JP2019036791A (en) * | 2017-08-10 | 2019-03-07 | キヤノン株式会社 | Image processing apparatus, image processing system, control method, and program |
| CN111405199A (en) * | 2020-03-27 | 2020-07-10 | 维沃移动通信（杭州）有限公司 | Image shooting method and electronic equipment |
-
2021
- 2021-10-12 CN CN202111187261.5A patent/CN113810725A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007221367A (en) * | 2006-02-15 | 2007-08-30 | Mitsubishi Electric Corp | Image transmission apparatus and system |
| CN105635624A (en) * | 2014-10-27 | 2016-06-01 | 华为技术有限公司 | Video image processing method, device and system |
| JP2019036791A (en) * | 2017-08-10 | 2019-03-07 | キヤノン株式会社 | Image processing apparatus, image processing system, control method, and program |
| CN109068052A (en) * | 2018-07-24 | 2018-12-21 | 努比亚技术有限公司 | video capture method, mobile terminal and computer readable storage medium |
| CN111405199A (en) * | 2020-03-27 | 2020-07-10 | 维沃移动通信（杭州）有限公司 | Image shooting method and electronic equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114520873A (en) * | 2021-12-31 | 2022-05-20 | 广州文远知行科技有限公司 | Sweeper video acceleration method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7242850B2 (en) | Frame-interpolated variable-rate motion imaging system | |
| CN111402399B (en) | Face driving and live broadcasting method and device, electronic equipment and storage medium | |
| KR101527672B1 (en) | System and method for video caption re-overlaying for video adaptation and retargeting | |
| US7852368B2 (en) | Method and apparatus for composing images during video communications | |
| US20160065862A1 (en) | Image Enhancement Based on Combining Images from a Single Camera | |
| WO2015024362A1 (en) | Image processing method and device | |
| CN113973190A (en) | Video virtual background image processing method and device and computer equipment | |
| CN112637517B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN111985281B (en) | Image generation model generation method and device and image generation method and device | |
| KR20170106389A (en) | Video transmission based on independently encoded background updates | |
| JP2022003797A (en) | Static video recognition | |
| CN111405339A (en) | Split screen display method, electronic equipment and storage medium | |
| JP7218786B2 (en) | Image processing device, image processing method and program | |
| CN113810725A (en) | Video processing method, device, storage medium and video communication terminal | |
| US20160142633A1 (en) | Capture apparatuses of video images | |
| Ekmekcioglu et al. | Visual attention model aided non-uniform asymmetric coding of stereoscopic video | |
| CN116962743A (en) | Video image coding and matting method and device and live broadcast system | |
| CN113938617A (en) | Multi-channel video display method and equipment, network camera and storage medium | |
| CN117478824B (en) | Conference video generation method and device, electronic equipment and storage medium | |
| CN113762156B (en) | Video data processing method, device and storage medium | |
| CN112887655B (en) | Information processing method and information processing device | |
| JP4506190B2 (en) | VIDEO DISPLAY DEVICE, VIDEO DISPLAY METHOD, VIDEO DISPLAY METHOD PROGRAM, AND RECORDING MEDIUM CONTAINING VIDEO DISPLAY METHOD PROGRAM | |
| US20240155095A1 (en) | Systems and methods for processing volumetric images | |
| CN114640882A (en) | Video processing method and device, electronic equipment and computer readable storage medium | |
| EP4052455A1 (en) | Automatic slow motion video recording |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |