CN114286194A - Instant messaging video processing method and device, electronic equipment and storage medium - Google Patents
Instant messaging video processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114286194A CN114286194A CN202111489734.7A CN202111489734A CN114286194A CN 114286194 A CN114286194 A CN 114286194A CN 202111489734 A CN202111489734 A CN 202111489734A CN 114286194 A CN114286194 A CN 114286194A
- Authority
- CN
- China
- Prior art keywords
- video
- stream data
- processing
- video stream
- video processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 28
- 238000012545 processing Methods 0.000 claims abstract description 217
- 238000000034 method Methods 0.000 claims abstract description 58
- 230000008569 process Effects 0.000 claims description 26
- 230000015654 memory Effects 0.000 claims description 24
- 230000007246 mechanism Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 13
- 230000000007 visual effect Effects 0.000 claims description 7
- 230000005484 gravity Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 230000000977 initiatory effect Effects 0.000 description 4
- 238000009877 rendering Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
Images
Abstract
The disclosure relates to a processing method, a device, an electronic device and a readable storage medium of an instant messaging video, wherein the method comprises the following steps: acquiring one or more video stream data under instant messaging, and determining target video stream data needing to execute video processing from the one or more video stream data; then according to the data processing capacity of the receiving end equipment and the video image quality requirement, a video processing strategy aiming at the target video stream data is obtained; and processing the target video stream data according to the video processing strategy to obtain the processed target video stream data. The method provided by the disclosure realizes the satisfaction of the user impression requirements, and in addition, the video processing strategy is determined by comprehensively considering the performance of the receiving end equipment and the video image quality requirements, so that the user impression requirements are maximally satisfied under the limited calculation power of the receiving end equipment.
Description
Technical Field
The present disclosure relates to the field of video processing technologies, and in particular, to a method and an apparatus for processing an instant messaging video, an electronic device, and a storage medium.
Background
With the rapid development of Real-Time Communication (RTC) technology, RTC scenes have been growing explosively, such as product live broadcast, teaching live broadcast, game live broadcast, and so on. The live broadcast of the RTC scene generally includes the flows of video stream data acquisition, encoding, transmission, decoding, rendering and playing. For example, a live broadcast initiating side (i.e., a sending end) starts a camera and collects video stream data in real time, the live broadcast initiating side encodes and transmits the collected video stream data to a service side, the service side transmits the video stream data to other associated terminal sides (i.e., receiving ends) in real time, and if the terminal side watches live broadcast, the terminal side decodes and renders the video stream data in real time.
However, due to the limitation of multiple factors such as network transmission bandwidth and real-time performance, the video quality at the receiving end is greatly different from that at the transmitting end, and the requirement of the user for the video quality when watching the live broadcast cannot be met.
Disclosure of Invention
In order to solve the technical problem or at least partially solve the technical problem, the present disclosure provides a method and an apparatus for processing an instant messaging video, an electronic device, and a storage medium.
In a first aspect, the present disclosure provides a method for processing an instant messaging video, including:
acquiring one or more paths of video stream data in instant messaging, and determining target video stream data needing to execute video processing from the one or more paths of video stream data;
acquiring a video processing strategy aiming at target video stream data according to the data processing capacity and video image quality requirements of receiving end equipment;
and processing the target video stream data according to the video processing strategy to obtain the processed target video stream data.
As a possible implementation manner, the video processing policy is used to indicate a plurality of video processing manners and/or a sequence of the plurality of video processing manners that need to be executed for the target video stream data.
As a possible implementation manner, the acquiring a video processing policy for target video stream data according to the data processing capability and the video quality requirement of the receiving end device includes:
determining the plurality of video processing modes according to the video quality requirement comprises: a first video processing mode and a second video processing mode;
if the data processing capacity of the receiving end equipment meets a preset condition, determining that the execution sequence of the first video processing mode is positioned before the second video processing mode;
and if the data processing capability of the receiving terminal equipment does not meet the preset condition, determining that the execution sequence of the second video processing mode is positioned before the first video processing mode.
In one possible embodiment, the first video processing method is a picture quality enhancement process, and the second video processing method is a scaling process.
As one possible implementation, the image quality enhancement processing includes: one or more of a sharpening enhancement process, a contrast enhancement process, and a color enhancement process.
As a possible implementation, the determining, from the one or more paths of video stream data, target video stream data that needs to perform video processing includes:
and determining the target video stream data from the one or more paths of video stream data according to the data processing capacity and/or a preset attention mechanism of the receiving end equipment.
As a possible implementation, the preset attention mechanism is used for representing a visual gravity center area in a picture displayed by the receiving end device; alternatively, the first and second electrodes may be,
the preset attention mechanism is used for characterizing video stream data sent by a specific client, and the one or more paths of video stream data comprise the video stream data sent by the specific client.
In a second aspect, the present disclosure provides an apparatus for processing an instant messaging video, including:
the acquisition module is used for acquiring one or more paths of video stream data in instant messaging;
the decision module is used for determining target video stream data needing to execute video processing from the one or more paths of video stream data; acquiring a video processing strategy aiming at target video stream data according to the data processing capacity and video image quality requirements of receiving end equipment;
and the execution module is used for processing the target video stream data according to the video processing strategy to obtain the processed target video stream data.
As a possible implementation manner, the video processing policy is used to indicate a plurality of video processing manners and/or a sequence of the plurality of video processing manners that need to be executed for the target video stream data.
As a possible implementation manner, the decision module is specifically configured to determine, according to the video quality requirement, the plurality of video processing manners, including: a first video processing mode and a second video processing mode; if the data processing capacity of the receiving end equipment meets a preset condition, determining that the execution sequence of the first video processing mode is positioned before the second video processing mode; and if the data processing capability of the receiving terminal equipment does not meet the preset condition, determining that the execution sequence of the second video processing mode is positioned before the first video processing mode.
In one possible embodiment, the first video processing method is a picture quality enhancement process, and the second video processing method is a scaling process.
As one possible implementation, the image quality enhancement processing includes: one or more of a sharpening enhancement process, a contrast enhancement process, and a color enhancement process.
As a possible implementation manner, the decision module is specifically configured to determine the target video stream data from the one or more paths of video stream data according to a data processing capability and/or a preset attention mechanism of the receiving end device.
As a possible implementation, the preset attention mechanism is used for representing a visual gravity center area in a picture displayed by the receiving end device; or the preset attention mechanism is used for characterizing video stream data sent by a specific client, and the one or more paths of video stream data comprise the video stream data sent by the specific client.
In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor;
the memory is configured to store computer program instructions;
the processor is configured to execute the computer program instructions to cause the electronic device to implement the method of processing instant messaging video according to any one of the first aspect.
In a fourth aspect, the present disclosure provides a readable storage medium comprising: computer program instructions; when executed by at least one processor of an electronic device, the computer program instructions cause the electronic device to implement the method for processing instant messaging video according to any one of the first aspect.
In a fifth aspect, the present disclosure provides a computer program product which, when executed by a computer, causes the computer to implement the processing method of an instant messaging video according to any one of the first aspect.
The present disclosure provides a processing method, a device, an electronic device and a storage medium for an instant messaging video, wherein the method comprises the following steps: acquiring one or more video stream data under instant messaging, and determining target video stream data needing to execute video processing from the one or more video stream data; then according to the data processing capacity of the receiving end equipment and the video image quality requirement, a video processing strategy aiming at the target video stream data is obtained; and processing the target video stream data according to the video processing strategy to obtain the processed target video stream data. The method provided by the disclosure realizes the satisfaction of the user impression requirements, and in addition, the video processing strategy is determined by comprehensively considering the performance of the receiving end equipment and the video image quality requirements, so that the user impression requirements are maximally satisfied under the limited calculation power of the receiving end equipment.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present disclosure, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic view of an application scenario of a processing method of an instant messaging video according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a processing method of an instant messaging video according to an embodiment of the present disclosure;
fig. 3A and 3B are schematic diagrams of an electronic screen of a receiving end device provided by the present disclosure;
fig. 4 is a flowchart of a processing method of an instant messaging video according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a device for processing an instant messaging video according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, aspects of the present disclosure will be further described below. It should be noted that the embodiments and features of the embodiments of the present disclosure may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced in other ways than those described herein; it is to be understood that the embodiments disclosed in the specification are only a few embodiments of the present disclosure, and not all embodiments.
In an RTC scenario, limited by multiple factors such as network transmission bandwidth and real-time performance, a sending end generally uses a lower code rate to perform encoding, and frame loss, packet loss and the like occur in a transmission process, so that the video quality of a receiving end is greatly reduced relative to that of the sending end, and the requirement of a user on the video quality when watching live broadcast cannot be met.
Based on the method, appropriate post-processing can be carried out on the video stream data at the receiving end so as to improve the video quality and further improve the viewing experience of the user.
Based on this, the instant messaging video processing method, the instant messaging video processing device, the electronic device and the storage medium provided by the disclosure are used for obtaining one or more paths of video stream data in instant messaging, comprehensively determining a video processing strategy according to factors such as data processing capacity of receiving end equipment, video image quality requirements and the like, and then performing video processing based on the video processing strategy, so that the user impression requirements are met.
The data processing capability of the receiving end device mentioned in the present disclosure may include, but is not limited to: the performance index of a central processing unit of the receiving end device, the performance index of a memory resource of the receiving end device, the version of a client used by the receiving end device in an RTC scene, and the like. The implementation manner of the receiving end device for acquiring the data processing capability is not limited in the present disclosure, for example, the receiving end device may acquire the data processing capability from an installed application program that monitors the operation state of the receiving end device.
The video quality requirement mentioned in the present disclosure is used for representing the quality requirement of a video frame image displayed on an electronic screen of a receiving end device in an RTC scene, where the video quality requirement may be preset or determined based on a setting of a user, and the present disclosure does not limit this. The video quality requirements may include, but are not limited to: sharpening, color, contrast, saturation, sharpness, etc. in one or more dimensions.
Fig. 1 is a schematic view of an application scenario of a processing method of an instant messaging video according to an embodiment of the present disclosure. Referring to the scenario 100 shown in fig. 1, it includes: a plurality of terminal devices 101 and a server device 102, the plurality of terminal devices 101 being capable of communicating with the server device 102.
The terminal device 101 may be a tablet computer, a mobile phone (e.g., a folding screen mobile phone, a large screen mobile phone, a smart phone, etc.), a wearable device, a vehicle-mounted device, an Augmented Reality (AR)/Virtual Reality (VR) device, a notebook computer, a super-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), a smart television, a smart screen, a high-definition television, a 4K television, and other internet of things (IOT) devices, and the present disclosure does not set any limit to specific types of electronic devices.
The server device 102 may be, for example, a cloud service platform, an internet of things platform, a business platform, or the like. The server device 102 may be a physical server, a cloud server, or a server cluster, which is not limited in this disclosure.
Exemplarily, it is assumed that a plurality of terminal devices 101 and a server device 102 shown in fig. 1 are currently performing product live broadcasting, where, in fig. 1, left and 2 terminal devices 101 are a main broadcasting terminal and a sub-main broadcasting terminal, that is, the left 2 terminal devices are all live broadcasting initiating sides, the right 2 terminal devices 101 in fig. 1 are live broadcasting watching user sides, and for the right 2 terminal devices 101 in fig. 1, video processing may be performed by executing the processing method of instant messaging video provided by the embodiment of the present disclosure, so as to meet a requirement of a user on video quality when watching product live broadcasting.
It should be noted that the scenario shown in fig. 1 is not limited to performing live broadcast of a product, and may also be used to implement other RTC scenarios, for example, a video call scenario, and the like; in addition, the number of the terminal devices 101 involved in the RTC scenario is not limited to 4 shown in fig. 1, and the number of the terminal devices 101 may be more or less, which is not limited in this disclosure.
The following embodiments are described by taking as an example a processing method for the receiving end device to execute the instant messaging video.
Fig. 2 is a flowchart of a processing method of an instant messaging video according to an embodiment of the present disclosure. Referring to fig. 2, the method provided by this embodiment includes:
s201, acquiring one or more paths of video stream data in the instant messaging.
The instant messaging can be a product live broadcast scene, a teaching live broadcast scene, a video call scene and the like. The video stream data is obtained in an RTC scene within a preset time length. The preset time period is not limited in the present disclosure, for example, the preset time period may be 3 seconds, 5 seconds, and the like.
For example, in a video call scenario, where the user 1, the user 2, and the user 3 are performing video calls, the obtained multiple paths of video stream data for the user 1 include: the method comprises the steps that a user 2 starts video stream data collected by a camera in real time and a user 3 starts video stream data collected by the camera in real time; similarly, for the user 2, the acquired multiple video stream data includes: the user 1 starts video stream data collected by the camera in real time and the user 3 starts video stream data collected by the camera in real time. The video stream data acquired by the user 1, the user 2 and the user 3 respectively starting the cameras in real time can be transmitted to other users in the scene through the server side equipment.
For another example, in a live broadcast scene of a product, the user 1 is a live broadcast initiating terminal, the user 2 is a live broadcast watching terminal, and for the user 2, the obtained video stream data includes video stream data acquired by the user 1 by starting a camera in real time. The video stream data collected in real time by the user 1 starting the camera can be transmitted to the user 2 through the server device.
S202, determining target video stream data needing to execute video processing from the one or more paths of video stream data.
The present disclosure is not limited to the implementation of determining the target video stream data. For example, the target video stream data to be subjected to the video processing may be determined based on the data processing capability of the sink device, or may be further determined in accordance with a preset policy in combination with the data processing capability of the sink device. The preset policy may be, but is not limited to: the present invention relates to a specific sending end device, a display layout of a receiving end device, and an adjustment operation of a user for the display of the receiving end device.
For example, when the data processing capability of the receiving end device is high, it may be satisfied that video processing is performed on each path of video stream data, and the target video stream data includes each path of video stream data acquired by the receiving end device; for another example, when the data processing capability of the receiving end device is low, it may be satisfied that video processing is performed on part of the video stream data, and then the part of the video stream data that needs to be subjected to video processing may be determined from the received video stream data according to a preset policy.
S203, acquiring a video processing strategy aiming at target video stream data according to the data processing capacity and video image quality requirements of the receiving terminal equipment.
The video processing strategy is used for indicating a plurality of video processing modes which need to be executed aiming at the target video stream data and the sequence of the plurality of video processing modes. It should be noted that, when a plurality of video processing modes are executed on video stream data, the requirements for the data processing capability of the receiving end device due to the different precedence orders of the plurality of video processing modes are also different, so that when the video processing strategy is obtained, the requirements for the data processing capability of the receiving end device due to the execution order of the plurality of video processing modes are considered as well as the video processing modes that need to be executed on the target video stream data, and the maximum improvement of the video image quality under the limited calculation power of the receiving end device is ensured as much as possible.
Exemplarily, the receiving end device may determine which video processing methods need to be adopted according to specific video image quality requirements; and then, determining the execution sequence of each video processing mode by combining the current data processing capacity of the receiving terminal equipment and the data processing capacity requirement of each video processing mode.
Illustratively, the plurality of video processing modes include: the first video processing method and the second video processing method are, for example, image quality enhancement processing, and the second video processing method is scaling processing, where the image quality enhancement processing may include one or more of sharpening enhancement processing, contrast enhancement processing, and color enhancement processing. The receiving end device may determine which processing mode of the image quality enhancement processing that needs to be executed includes sharpening enhancement processing, contrast enhancement processing, and color enhancement processing according to the video image quality requirement, and then determine the execution sequence of the image quality enhancement processing and the scaling processing according to the data processing capability of the receiving end device.
It should be noted that, in one case, the image quality enhancement processing may be performed first, and then the scaling processing may be performed, which is simple to operate and convenient to implement, and since the scaling processing may be implemented at the rendering and displaying stage, the processing efficiency may be greatly improved in the process of actually performing video combing by the receiving end device; in another case, the scaling process may be performed first, and then the image quality enhancement process may be performed, so that the image quality of the video may be ensured.
Comparing the two cases, the video processing efficiency of the first case is high, but the quality of the video image finally rendered and displayed on the electronic screen of the receiving end device is lost relative to the video frame image after the image quality enhancement processing is performed; the second case is high in video quality, and if the data processing capability of the receiving end device is the same, the overall processing efficiency of the second case is lower than that of the first case. Therefore, in practical application, the execution sequence of each video processing mode included in the video processing strategy can be flexibly determined according to the data processing capability of the receiving terminal equipment.
And S204, processing the target video stream data according to the video processing strategy to obtain the processed target video stream data.
Specifically, the receiving end device may sequentially perform corresponding video processing on the target video stream data according to the multiple video processing manners indicated by the video processing policy and the sequence of the multiple video processing manners, to obtain the processed target video stream data.
For example, the first video processing method is sharpening enhancement processing, and the second video processing method is scaling processing.
Case 1:
the receiving end equipment carries out sharpening enhancement processing on each video frame image included in the target video stream data to obtain each processed video frame image; and then, zooming each video frame image after sharpening enhancement processing according to the display resolution set by the user, and rendering and displaying each zoomed video frame image on a display screen of the receiving end equipment for the user to watch.
Case 2:
the receiving end equipment performs zooming processing on each video frame image included in the target video stream data according to the display resolution set by the user to obtain each zoomed video frame image; and then, carrying out sharpening enhancement processing on the zoomed video frame images, and rendering and displaying the video frame images subjected to sharpening enhancement processing on a display screen of receiving end equipment for a user to watch.
It should be noted that the "display resolution set by the user" mentioned in case 1 and case 2 indicates the display size of the video frame image on the electronic screen of the receiving-end device. This is illustrated by way of example in fig. 3A and 3B. Fig. 3A and 3B exemplarily show user interface diagrams in two RTC scenarios.
Referring to fig. 3A, it is assumed that the RTC scene shown in fig. 3A is a live product scene, where a user 1 is a main broadcast, a user 2 is a sub-main broadcast, a receiving end device is an electronic device used by the user 3, and an electronic screen of the receiving end device includes a window 301a and a window 302a, where the window 301a is used to show video stream data collected by the user 1 (i.e., the main broadcast end) in real time, and the window 302a is used to show video stream data collected by the user 2 (i.e., the sub-main broadcast end) in real time. Since the size of the window 301a may be different from the size of each video frame image in the video stream data collected by the main broadcaster in real time, and similarly, the size of the window 302a may be different from the size of each video frame image in the video stream data collected by the subsidiary main broadcaster in real time, scaling processing is required to match the size of the video frame image collected by the main broadcaster in real time with the size of the window 301a, and to match the size of the video frame image collected by the subsidiary main broadcaster in real time with the size of the window 302 a.
Therefore, in the scene shown in fig. 3A, for the video stream data sent by the anchor side, the display resolution set by the corresponding user is the resolution corresponding to the window 301 a; for the video stream data sent by the secondary main broadcast end, the display resolution set by the corresponding user is the resolution corresponding to the window 302 a.
Referring to fig. 3B, it is assumed that the RTC scene shown in fig. 3B is a scene in which the user 1, the user 2, the user 3, and the user 4 perform a video call, where the receiving end device is an electronic device used by any one of the user 1 to the user 4, and during the video call, an electronic screen of the receiving end device includes windows 301B to 304B, where the windows 301B to 304B are respectively used for displaying video stream data acquired by the user 1 to the user 4 in real time. Since the sizes of the windows 301b to 304b may match the sizes of the video frames in the video stream data captured by the users 1 to 4 respectively in real time, scaling is required to match the sizes of the video frame images captured by the users in real time with the sizes of the corresponding windows.
Therefore, in the scenario shown in fig. 3B, for the video stream data sent by the user 1, the display resolution set by the corresponding user is the resolution corresponding to the window 301B; for the video stream data sent by the user 2, the display resolution set by the corresponding user is the resolution corresponding to the window 302 b; for the video stream data sent by the user 3, the display resolution set by the corresponding user is the resolution corresponding to the window 303 b; for the video stream data sent by the user 4, the display resolution set by the corresponding user is the resolution corresponding to the window 304 b.
According to the method provided by the embodiment, after one or more paths of video stream data in instant messaging are acquired, a video processing strategy is comprehensively determined according to factors such as data processing capacity of receiving end equipment and video image quality requirements, and then video processing is carried out based on the video processing strategy, so that the user impression requirements are met.
Fig. 4 is a flowchart of a processing method of an instant messaging video according to an embodiment of the present disclosure. Referring to fig. 4, the method provided in this embodiment includes:
s401, one or more paths of video stream data in the instant messaging are obtained.
Step S401 in this embodiment is similar to step S201 in the embodiment shown in fig. 2, and reference may be made to the detailed description of the embodiment shown in fig. 2, which is not repeated herein for brevity.
In conjunction with the embodiment shown in fig. 2, step S202 can be implemented by step S402 in this embodiment.
S402, determining the target video stream data from the one or more paths of video stream data according to the data processing capacity and/or the preset attention mechanism of the receiving end equipment.
Wherein, according to the data processing capability of the receiving end device, the target video stream data is determined, as described in detail in S202 in the foregoing embodiment shown in fig. 2.
In this embodiment, how to determine target video stream data based on a preset attention mechanism is described in detail, where it should be noted that the preset attention mechanism is used to represent a visual center of gravity area in a picture displayed by a receiving end device; or, the preset attention mechanism is used for characterizing video stream data sent by a specific client, and the one or more paths of video stream data comprise the video stream data sent by the specific client.
In conjunction with the RTC scene shown in fig. 3A, in general, the attention of the user 3 is focused on the window 1 corresponding to the main broadcast end, that is, the window 1 is a visual center-of-gravity area in the picture displayed by the receiving end device, and then the target video stream data can be determined to be the video stream data sent by the main broadcast end according to the visual center-of-gravity area (i.e., the window 1) in the picture displayed by the receiving end device.
In conjunction with the RTC scenario shown in fig. 3B, assuming that the receiving device is a device used by the user 1, in a normal case, the user 1 focuses on the video frame images corresponding to the users 2 to 4, and therefore, it can be determined that the target video stream data may include video stream data respectively transmitted by 1 or more users among the users 2 to 4.
It should be noted that, the target video stream data mentioned herein may include video stream data sent by each of the users 2 to 4, which may be further determined by combining the data processing capability of the current receiving end device. For example, if the data processing capability of the current receiving end device is high and can satisfy the requirement of performing video processing on video stream data respectively sent by the users 2 to 4, the target video stream data may include video stream data respectively sent by the users 2 to 4; if the data processing capability of the current receiving end device is low, it may be satisfied that video processing is performed on video stream data sent by some users from the user 2 to the user 4, and the target video stream data may include video stream data sent by some users from the user 2 to the user 4, where the target video stream data may be determined according to frame loss and packet loss conditions of the video stream data sent by the user 2 to the user 4, and the video stream data with serious frame loss and packet loss is determined as the target video stream data.
Based on the embodiment shown in fig. 3B, in some cases, the receiving end device supports adjusting the size of each window of the picture displayed in the electronic screen according to the trigger operation of the user. For example, on the basis of the embodiment shown in fig. 3B, the user 1 inputs a trigger operation (e.g., a click operation) for the window 302B, and the receiving-end device may display only the video frame image of the window 302B on the electronic screen, and the window 302B becomes larger in size compared to the previous window. In this case, it is indicated that the user 1 wants to watch the video stream data sent by the user 2 with emphasis, it may be determined that the target video stream data includes the video stream data sent by the user 2 based on the trigger operation of the user, and the video stream data sent by the user 3 and the user 4 respectively may not be displayed. If the receiving end device returns to the state shown in fig. 3B according to the trigger operation of the user, it is determined again that the target video stream data includes video stream data respectively sent by 1 or more users from the user 2 to the user 4.
And S403, acquiring a video processing strategy aiming at target video stream data according to the data processing capacity and video image quality requirement of the receiving terminal equipment.
S404, processing the target video stream data according to the video processing strategy to obtain the processed target video stream data.
In this embodiment, steps S403 and S404 are similar to steps S203 and S204 in the embodiment shown in fig. 2, respectively, and reference may be made to the detailed description of the embodiment shown in fig. 2, so that for brevity, no further description is provided here.
According to the method provided by the embodiment, after one or more paths of video stream data in instant messaging are acquired, a video processing strategy is comprehensively determined according to factors such as data processing capacity of receiving end equipment and video image quality requirements, and then video processing is carried out based on the video processing strategy, so that the user impression requirements are met. In addition, the target video stream data can be determined based on the data processing capability of the receiving end device and a preset attention mechanism, so that video processing is performed on the image of the area with high attention of the user, and the user impression can be further improved.
The disclosure also provides an instant messaging video processing device.
Fig. 5 is a schematic structural diagram of a device for processing an instant messaging video according to an embodiment of the present disclosure. Referring to fig. 5, the instant messaging video processing apparatus 500 provided in this embodiment includes:
the obtaining module 501 is configured to obtain one or more paths of video stream data in the instant messaging.
A decision module 502, configured to determine, from the one or more paths of video stream data, target video stream data that needs to perform video processing; and acquiring a video processing strategy aiming at target video stream data according to the data processing capacity of the receiving terminal equipment and the video image quality requirement.
The executing module 503 is configured to process the target video stream data according to the video processing policy, and acquire the processed target video stream data.
As a possible implementation manner, the video processing policy is used to indicate a plurality of video processing manners and/or a sequence of the plurality of video processing manners that need to be executed for the target video stream data.
As a possible implementation manner, the decision module 502 is specifically configured to determine, according to the video quality requirement, the multiple video processing manners including: a first video processing mode and a second video processing mode; if the data processing capacity of the receiving end equipment meets a preset condition, determining that the execution sequence of the first video processing mode is positioned before the second video processing mode; and if the data processing capability of the receiving terminal equipment does not meet the preset condition, determining that the execution sequence of the second video processing mode is positioned before the first video processing mode.
In one possible embodiment, the first video processing method is a picture quality enhancement process, and the second video processing method is a scaling process.
As one possible implementation, the image quality enhancement processing includes: one or more of a sharpening enhancement process, a contrast enhancement process, and a color enhancement process.
As a possible implementation manner, the decision module 502 is specifically configured to determine the target video stream data from the one or more paths of video stream data according to a data processing capability and/or a preset attention mechanism of the receiving end device.
As a possible implementation, the preset attention mechanism is used for representing a visual gravity center area in a picture displayed by the receiving end device; or the preset attention mechanism is used for characterizing video stream data sent by a specific client, and the one or more paths of video stream data comprise the video stream data sent by the specific client.
The processing apparatus for instant messaging video provided in this embodiment may be configured to execute the technical solution of any of the foregoing method embodiments, and the implementation principle and the technical effect thereof are similar.
Exemplarily, the present disclosure also provides an electronic device.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. Referring to fig. 6, the electronic device 600 provided in the present embodiment includes: a memory 601 and a processor 602. The memory 601 may be a separate physical unit, and may be connected to the processor 602 through a bus 603. The memory 601 and the processor 602 may also be integrated, implemented by hardware, and the like.
The memory 601 is used for storing program instructions, and the processor 602 calls the program instructions to execute the technical solution of any one of the above method embodiments.
Alternatively, when part or all of the methods of the above embodiments are implemented by software, the electronic device 600 may only include the processor 602. A memory 601 for storing programs is located outside the electronic device 600, and a processor 602 is connected to the memory via circuits/wires for reading and executing the programs stored in the memory.
The processor 602 may be a Central Processing Unit (CPU), a Network Processor (NP), or a combination of a CPU and an NP.
The processor 602 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.
The memory 601 may include a volatile memory (volatile memory), such as a random-access memory (RAM); the memory may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a Hard Disk Drive (HDD) or a solid-state drive (SSD); the memory may also comprise a combination of memories of the kind described above.
An embodiment of the present disclosure also provides a readable storage medium, including: computer program instructions; the computer program instructions, when executed by at least one processor of the electronic device, implement the method for processing instant messaging video as shown in any of the above method embodiments.
The embodiment of the present disclosure further provides a computer program product, and when the computer program product is executed by a computer, the computer is enabled to implement the processing method of the instant messaging video shown in any method embodiment.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The foregoing are merely exemplary embodiments of the present disclosure, which enable those skilled in the art to understand or practice the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A processing method of instant messaging video is characterized by comprising the following steps:
acquiring one or more paths of video stream data in instant messaging, and determining target video stream data needing to execute video processing from the one or more paths of video stream data;
acquiring a video processing strategy aiming at target video stream data according to the data processing capacity and video image quality requirements of receiving end equipment;
and processing the target video stream data according to the video processing strategy to obtain the processed target video stream data.
2. The method according to claim 1, wherein the video processing policy is used to indicate a plurality of video processing manners and/or a sequence of the plurality of video processing manners that need to be executed for the target video stream data.
3. The method according to claim 2, wherein the obtaining a video processing policy for the target video stream data according to the data processing capability and video quality requirement of the receiving device comprises:
determining the plurality of video processing modes according to the video quality requirement comprises: the method comprises the following steps of (1) carrying out a first video processing mode and a second video processing mode, wherein the first video processing mode is image quality enhancement processing, and the second video processing mode is zooming processing;
if the data processing capacity of the receiving end equipment meets a preset condition, determining that the execution sequence of the first video processing mode is positioned before the second video processing mode;
and if the data processing capability of the receiving terminal equipment does not meet the preset condition, determining that the execution sequence of the second video processing mode is positioned before the first video processing mode.
4. The method according to claim 3, wherein the image quality enhancement process comprises: one or more of a sharpening enhancement process, a contrast enhancement process, and a color enhancement process.
5. The method according to any one of claims 1 to 4, wherein the determining, from the one or more video stream data, target video stream data that needs to be subjected to video processing comprises:
and determining the target video stream data from the one or more paths of video stream data according to the data processing capacity and/or a preset attention mechanism of the receiving end equipment.
6. The method of claim 5,
the preset attention mechanism is used for representing a visual gravity center area in a picture displayed by the receiving end equipment; alternatively, the first and second electrodes may be,
the preset attention mechanism is used for characterizing video stream data sent by a specific client, and the one or more paths of video stream data comprise the video stream data sent by the specific client.
7. An apparatus for processing instant messaging video, comprising:
the acquisition module is used for acquiring one or more paths of video stream data in instant messaging;
the decision module is used for determining target video stream data needing to execute video processing from the one or more paths of video stream data; acquiring a video processing strategy aiming at target video stream data according to the data processing capacity and video image quality requirements of receiving end equipment;
and the execution module is used for processing the target video stream data according to the video processing strategy to obtain the processed target video stream data.
8. The apparatus according to claim 7, wherein the video processing policy is configured to indicate a plurality of video processing manners and/or a sequence of the plurality of video processing manners that need to be performed with respect to the target video stream data.
9. An electronic device, comprising: a memory and a processor;
the memory is configured as a memory configured to store computer program instructions;
the processor is configured to execute the computer program instructions to cause the electronic device to implement the method of processing instant messaging video according to any one of claims 1 to 6.
10. A readable storage medium, comprising: computer program instructions; the computer program instructions, when executed by at least one processor of an electronic device, cause the electronic device to implement the method of processing instant messaging video of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111489734.7A CN114286194B (en) | 2021-12-08 | Instant messaging video processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111489734.7A CN114286194B (en) | 2021-12-08 | Instant messaging video processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114286194A true CN114286194A (en) | 2022-04-05 |
| CN114286194B CN114286194B (en) | 2024-05-03 |
Family
ID=
Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101031083A (en) * | 2007-04-04 | 2007-09-05 | 中山大学 | Method for realizing realtime video transmission by wavelet |
| CN102685415A (en) * | 2011-03-18 | 2012-09-19 | 华为终端有限公司 | Method, system, device and terminal for processing video picture |
| CN103517137A (en) * | 2013-09-30 | 2014-01-15 | 广州华多网络科技有限公司 | Network video transmission method and system, sending/receiving end processing method and device |
| CN104281427A (en) * | 2014-03-10 | 2015-01-14 | 深圳深讯和科技有限公司 | Video data processing method and system in interactive application |
| WO2016165375A1 (en) * | 2015-09-11 | 2016-10-20 | 中兴通讯股份有限公司 | Screen projection video data transmission method, device and system |
| WO2017101355A1 (en) * | 2015-12-18 | 2017-06-22 | 乐视控股（北京）有限公司 | Method and device for image processing |
| CN106993199A (en) * | 2017-04-28 | 2017-07-28 | 暴风集团股份有限公司 | A kind of method and system of real-time video transcoding |
| CN109379625A (en) * | 2018-11-27 | 2019-02-22 | Oppo广东移动通信有限公司 | Method for processing video frequency, device, electronic equipment and computer-readable medium |
| CN110401875A (en) * | 2018-04-24 | 2019-11-01 | 深信服科技股份有限公司 | A kind of virtual desktop video pictures method of quality control, system and equipment |
| CN110401820A (en) * | 2019-08-15 | 2019-11-01 | 北京迈格威科技有限公司 | Multipath video processing method, device, medium and electronic equipment |
| CN110446062A (en) * | 2019-07-18 | 2019-11-12 | 平安科技（深圳）有限公司 | Receiving handling method, electronic device and the storage medium of large data files transmission |
| CN110601872A (en) * | 2019-08-06 | 2019-12-20 | 联想(北京)有限公司 | Transmission method and device and storage medium |
| CN110662100A (en) * | 2018-06-28 | 2020-01-07 | 中兴通讯股份有限公司 | Information processing method, device and system and computer readable storage medium |
| CN110856044A (en) * | 2019-11-19 | 2020-02-28 | 深圳市迅雷网络技术有限公司 | Video image quality enhancement transmission method and related device |
| US20200169767A1 (en) * | 2018-11-27 | 2020-05-28 | Peter Hackes | Systems, Methods And Computer Program Products For Delivering Audio And Video Data Through Multiplexed Data Streams |
| CN111225238A (en) * | 2018-11-26 | 2020-06-02 | 华为技术有限公司 | Information processing method and related equipment |
| CN111263188A (en) * | 2020-02-17 | 2020-06-09 | 腾讯科技（深圳）有限公司 | Video image quality adjusting method and device, electronic equipment and storage medium |
| WO2020164532A1 (en) * | 2019-02-13 | 2020-08-20 | 华为技术有限公司 | Method, apparatus, and system for determining harq feedback resources |
| CN111787362A (en) * | 2020-06-18 | 2020-10-16 | 北京字节跳动网络技术有限公司 | Message processing method and device |
| CN112312229A (en) * | 2020-10-27 | 2021-02-02 | 唐桥科技(杭州)有限公司 | Video transmission method and device, electronic equipment and storage medium |
| CN112492395A (en) * | 2020-11-30 | 2021-03-12 | 维沃移动通信有限公司 | Data processing method and device and electronic equipment |
| CN113014858A (en) * | 2021-03-05 | 2021-06-22 | 深圳壹秘科技有限公司 | Method, system and device for changing resolution |
| CN113286405A (en) * | 2021-04-30 | 2021-08-20 | 深圳市凯润智能照明有限公司 | System and method for realizing synchronous control of lamplight by audio frequency and image |
| CN113301295A (en) * | 2021-04-08 | 2021-08-24 | 上海艾策通讯科技股份有限公司 | Multi-picture video data processing method, device, computer equipment and storage medium |
| CN113411218A (en) * | 2021-06-22 | 2021-09-17 | 北京金山云网络技术有限公司 | Method and device for evaluating instant messaging quality and electronic equipment |
Patent Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101031083A (en) * | 2007-04-04 | 2007-09-05 | 中山大学 | Method for realizing realtime video transmission by wavelet |
| CN102685415A (en) * | 2011-03-18 | 2012-09-19 | 华为终端有限公司 | Method, system, device and terminal for processing video picture |
| CN103517137A (en) * | 2013-09-30 | 2014-01-15 | 广州华多网络科技有限公司 | Network video transmission method and system, sending/receiving end processing method and device |
| CN104281427A (en) * | 2014-03-10 | 2015-01-14 | 深圳深讯和科技有限公司 | Video data processing method and system in interactive application |
| WO2016165375A1 (en) * | 2015-09-11 | 2016-10-20 | 中兴通讯股份有限公司 | Screen projection video data transmission method, device and system |
| WO2017101355A1 (en) * | 2015-12-18 | 2017-06-22 | 乐视控股（北京）有限公司 | Method and device for image processing |
| CN106993199A (en) * | 2017-04-28 | 2017-07-28 | 暴风集团股份有限公司 | A kind of method and system of real-time video transcoding |
| CN110401875A (en) * | 2018-04-24 | 2019-11-01 | 深信服科技股份有限公司 | A kind of virtual desktop video pictures method of quality control, system and equipment |
| CN110662100A (en) * | 2018-06-28 | 2020-01-07 | 中兴通讯股份有限公司 | Information processing method, device and system and computer readable storage medium |
| CN111225238A (en) * | 2018-11-26 | 2020-06-02 | 华为技术有限公司 | Information processing method and related equipment |
| CN109379625A (en) * | 2018-11-27 | 2019-02-22 | Oppo广东移动通信有限公司 | Method for processing video frequency, device, electronic equipment and computer-readable medium |
| US20200169767A1 (en) * | 2018-11-27 | 2020-05-28 | Peter Hackes | Systems, Methods And Computer Program Products For Delivering Audio And Video Data Through Multiplexed Data Streams |
| WO2020164532A1 (en) * | 2019-02-13 | 2020-08-20 | 华为技术有限公司 | Method, apparatus, and system for determining harq feedback resources |
| CN110446062A (en) * | 2019-07-18 | 2019-11-12 | 平安科技（深圳）有限公司 | Receiving handling method, electronic device and the storage medium of large data files transmission |
| CN110601872A (en) * | 2019-08-06 | 2019-12-20 | 联想(北京)有限公司 | Transmission method and device and storage medium |
| CN110401820A (en) * | 2019-08-15 | 2019-11-01 | 北京迈格威科技有限公司 | Multipath video processing method, device, medium and electronic equipment |
| CN110856044A (en) * | 2019-11-19 | 2020-02-28 | 深圳市迅雷网络技术有限公司 | Video image quality enhancement transmission method and related device |
| CN111263188A (en) * | 2020-02-17 | 2020-06-09 | 腾讯科技（深圳）有限公司 | Video image quality adjusting method and device, electronic equipment and storage medium |
| CN111787362A (en) * | 2020-06-18 | 2020-10-16 | 北京字节跳动网络技术有限公司 | Message processing method and device |
| CN112312229A (en) * | 2020-10-27 | 2021-02-02 | 唐桥科技(杭州)有限公司 | Video transmission method and device, electronic equipment and storage medium |
| CN112492395A (en) * | 2020-11-30 | 2021-03-12 | 维沃移动通信有限公司 | Data processing method and device and electronic equipment |
| CN113014858A (en) * | 2021-03-05 | 2021-06-22 | 深圳壹秘科技有限公司 | Method, system and device for changing resolution |
| CN113301295A (en) * | 2021-04-08 | 2021-08-24 | 上海艾策通讯科技股份有限公司 | Multi-picture video data processing method, device, computer equipment and storage medium |
| CN113286405A (en) * | 2021-04-30 | 2021-08-20 | 深圳市凯润智能照明有限公司 | System and method for realizing synchronous control of lamplight by audio frequency and image |
| CN113411218A (en) * | 2021-06-22 | 2021-09-17 | 北京金山云网络技术有限公司 | Method and device for evaluating instant messaging quality and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| EIHAB SAATIALSORUJI; SHIKHARESH MAJUMDAR: "A Video Segmentation Strategy for Video Processing Applications on Hadoop Clusters", 《2018 IEEE SMARTWORLD, UBIQUITOUS INTELLIGENCE & COMPUTING, ADVANCED & TRUSTED COMPUTING, SCALABLE COMPUTING & COMMUNICATIONS, CLOUD & BIG DATA COMPUTING, INTERNET OF PEOPLE AND SMART CITY INNOVATION (SMARTWORLD/SCALCOM/UIC/ATC/CBDCOM/IOP/SCI)》, 31 December 2018 (2018-12-31) * |
| 林镜华;雷为民;白松;李凌南;: "支持IPTV高清码流连续平滑播放的多级缓冲区设计与实现", 小型微型计算机系统, no. 05, 15 May 2010 (2010-05-15) * |
| 高贝;: "浅析网络高清存储的应用技术", 中国公共安全(综合版), no. 16, 15 October 2012 (2012-10-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108124194B (en) | Video live broadcast method and device and electronic equipment | |
| CN108737882B (en) | Image display method, image display device, storage medium and electronic device | |
| JP6326411B2 (en) | Video communication method, video communication apparatus, program, and recording medium | |
| CN106534757B (en) | Face exchange method and device, anchor terminal and audience terminal | |
| CN110392306B (en) | Data processing method and equipment | |
| CN110930467A (en) | Image processing method, electronic device and readable storage medium | |
| CN111787398A (en) | Video compression method, device, equipment and storage device | |
| CN112911318A (en) | Live broadcast room background replacement method and device, electronic equipment and storage medium | |
| Adeyemi-Ejeye et al. | Impact of packet loss on 4K UHD video for portable devices | |
| CN113992883A (en) | Video conference processing method, processing device, conference system, and storage medium | |
| CN110913118A (en) | Video processing method, device and storage medium | |
| CN106664432B (en) | Multimedia information playing method and system, acquisition equipment and standardized server | |
| CN111263113B (en) | Data packet sending method and device and data packet processing method and device | |
| CN110996137B (en) | Video processing method and device | |
| CN114286194B (en) | Instant messaging video processing method and device, electronic equipment and storage medium | |
| CN114286194A (en) | Instant messaging video processing method and device, electronic equipment and storage medium | |
| CN110662071A (en) | Video decoding method and apparatus, storage medium, and electronic apparatus | |
| CN114143616A (en) | Target video processing method and system, storage medium and electronic device | |
| CN108933945B (en) | GIF picture compression method, device and storage medium | |
| US10764578B2 (en) | Bit rate optimization system and method | |
| JP2004056662A (en) | Device, method and program for repeating image attached mail | |
| CN113141352A (en) | Multimedia data transmission method and device, computer equipment and storage medium | |
| CN112261441A (en) | Video data processing method, system, device and storage medium | |
| CN112929694A (en) | Video splicing method and device, storage medium and computer equipment | |
| KR20160136163A (en) | Apparatus and method for providing video communication service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |