WO2016202285A1

WO2016202285A1 - Real-time video transmission method and electronic apparatus

Info

Publication number: WO2016202285A1
Application number: PCT/CN2016/086134
Authority: WO
Inventors: 武俊敏
Original assignee: 美国掌赢信息科技有限公司; 武俊敏
Priority date: 2015-06-19
Filing date: 2016-06-17
Publication date: 2016-12-22

Abstract

The invention relates to the field of video technology. Disclosed are a real-time video transmission method and an electronic apparatus comprising: acquiring, from a current real-time video frame, a region of interest (ROI); if a network bandwidth meets a first preset condition, transmitting, according to a first transmission strategy, the current real-time video frame comprising the ROI; if the network bandwidth meets a second preset condition, transmitting, according to a second transmission strategy, the current real-time video frame comprising the ROI; if the network bandwidth meets a third preset condition, transmitting, according to a third transmission strategy, the current real-time video frame comprising the ROI; and if the network bandwidth meets a fourth preset condition, transmitting, according to a fourth transmission strategy, the current real-time video frame comprising the ROI. According to different network bandwidths, different transmission strategies are adopted for a real-time video so as to facilitate utilization of the network bandwidths and save network resources, thereby increasing an efficiency in transmitting the real-time video and improving user experience.

Description

Instant video transmission method and electronic device

Technical field

The present invention relates to the field of video, and in particular, to a method and a device for transmitting instant video.

Background technique

With the popularization of mobile devices and instant video communication technologies, more and more users choose to use mobile devices to realize real-time video interaction, which necessitates providing an instant video transmission method to realize real-time video transmission to meet the growing user demand. Instant video interaction needs.

The prior art provides a method for transmitting an instant video. The method includes: acquiring an instant video input by a user; encoding a uniform encoding manner for all areas in the instant video frame; and transmitting the encoded real-time video.

Due to the limitation and instability of the network bandwidth, when the method provided by the prior art is used, when the network bandwidth is low, the efficiency of the instant video transmission is reduced, and the transmission of the instant video picture is stopped, and the video is instantaneous. The connection is disconnected, which reduces the user experience during instant video interaction.

Summary of the invention

In order to improve the transmission efficiency of the instant video and improve the user experience, the present invention provides an instant video transmission method and an electronic device. The technical solution is as follows:

In a first aspect, a method for transmitting an instant video is provided, the method comprising:

Obtaining an area of interest in the current instant video frame;

Perform any of the following operations based on the preset conditions that the network bandwidth meets and the transmission policies supported by the electronic device:

If the network bandwidth meets the first preset condition, transmitting the current instant video frame including the interest area according to the first transmission policy; or

If the network bandwidth meets the second preset condition, transmitting the current instant video frame including the interest area according to the second transmission policy; or

If the network bandwidth meets the third preset condition, transmitting the current instant video frame including the interest area according to the third transmission policy; or

If the network bandwidth meets the fourth preset condition, transmitting the current instant video frame including the interest area according to the fourth transmission policy.

With reference to the first aspect, in a first possible implementation manner, the acquiring an area of interest in the current instant video frame includes:

Obtaining a first feature parameter for indicating the region of interest;

Obtaining the region of interest from the current instant video frame according to the first feature parameter.

With reference to the first aspect, or the first possible implementation manner of the first aspect, in a second possible implementation manner, the transmitting, according to the first transmission policy, the current instant video frame that includes the region of interest includes :

And processing the current instant video according to an importance matrix corresponding to the current instant video frame, and acquiring the processed instant video frame;

Transmitting the processed instant video frame.

With reference to the second possible implementation of the first aspect, in a third possible implementation, the current real-time video is processed according to an importance matrix corresponding to the current instant video frame, and the processing is performed. The following instant video frames include:

Generating an importance matrix corresponding to the current instant video frame according to the region of interest;

And processing, according to the importance matrix, the current instant video frame, acquiring the processed instant video frame, and inputting the processed instant video frame into the video encoder.

With reference to the third possible implementation manner of the foregoing aspect, in a fourth possible implementation, the generating an importance matrix corresponding to the current instant video frame according to the region of interest includes:

Setting an importance metric value of all pixels in the current instant video frame;

Performing low-pass filtering processing on the region of interest to generate a processed current instant video frame;

Generating the importance matrix corresponding to the current instant video frame according to the importance metric of all the pixels in the current current video frame.

With reference to the first aspect, the fourth possible implementation of the fourth aspect, in a fifth possible implementation, the transmitting the current instant video frame that includes the region of interest according to the second transmission policy Strategies include:

Dividing an instant video frame including at least the region of interest into at least one macroblock, and setting a quantization parameter corresponding to the at least one macroblock;

Encoding the at least one macroblock according to the quantization parameter to generate an encoded instant video frame;

Transmitting the encoded instant video frame.

With reference to the fifth possible implementation manner of the foregoing aspect, in a sixth possible implementation, the setting, by the setting, the quantization parameter corresponding to the at least one macroblock includes:

Dividing the processed current instant video frame into at least one macroblock;

Setting a quantization parameter of a macroblock including all or part of the region of interest as a first value;

A quantization parameter that sets a macroblock that does not include all or part of the region of interest is a second value.

With reference to the first aspect to the sixth possible implementation of the first aspect, in a seventh possible implementation, the transmitting, according to the third transmission policy, the current instant video frame that includes the region of interest includes:

Obtaining residual data between the current instant video frame and the reference frame including the region of interest;

Transmitting the residual data.

With reference to the seventh possible implementation manner of the foregoing aspect, in an eighth possible implementation manner, the acquiring the residual data between the current instant video frame and the reference frame that includes the region of interest includes:

Obtaining a position parameter and a posture parameter of the interest area;

Normalizing the position parameter and the posture parameter to generate a standard position parameter and a standard Quasi-pose parameter

Obtaining residual data between the current instant video frame and the reference frame including the region of interest according to the standard location parameter and the standard gesture parameter.

With reference to the first aspect, in a ninth possible implementation manner, the transmitting, according to the fourth transmission policy, the current instant video frame that includes the region of interest includes:

Obtaining a second feature parameter for describing the region of interest;

Transmitting the second characteristic parameter.

In conjunction with the ninth possible implementation manner of the first aspect, in the tenth possible implementation manner, the acquiring, by using the second feature parameter, the second feature parameter is:

Performing triangulation on the region of interest to acquire at least one triangular region;

Acquiring the attitude parameter and the position parameter corresponding to the at least one triangular area;

Generating the second feature parameter according to the attitude parameter and the position parameter corresponding to the at least one triangular region.

In conjunction with the tenth possible implementation manner of the first aspect, in the eleventh possible implementation manner, the acquiring the posture parameter and the location parameter of the at least one triangular region includes:

Obtaining position parameters and attitude parameters for describing three feature points of any one of the triangular regions;

Generating a posture parameter and a position parameter corresponding to the arbitrary one of the triangular regions according to the position parameter and the posture parameter of the three feature points;

And so on, acquiring the attitude parameter and the position parameter corresponding to the at least one triangular region.

In a second aspect, an electronic device is provided, the electronic device comprising:

Obtaining a module, acquiring an area of interest in the current instant video frame;

a determining module, configured to determine whether the network bandwidth meets any one of a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition;

The decision module is further configured to trigger any one of the following modules according to a preset condition that the network bandwidth meets and a transmission policy supported by the electronic device:

a first processing module, configured to: according to the first transmission, when the network bandwidth meets a first preset condition a policy of transmitting the current instant video frame including the region of interest; or

a second processing module, configured to: when the network bandwidth meets a second preset condition, transmit the current instant video frame that includes the interest area according to a second transmission policy; or

a third processing module, configured to: when the network bandwidth meets a third preset condition, transmit the current instant video frame including the interest area according to a third transmission policy; or

And a fourth processing module, configured to: when the network bandwidth meets the fourth preset condition, transmit the current instant video frame including the interest area according to the fourth transmission policy.

With reference to the second aspect, in a first possible implementation manner, the acquiring module is specifically configured to:

Obtaining a first feature parameter for indicating the region of interest;

With reference to the second aspect or the first possible implementation manner of the second aspect, in a second possible implementation manner, the first processing module is specifically configured to:

Transmitting the processed instant video frame.

In conjunction with the second possible implementation of the second aspect, in a third possible implementation, the first processing module is specifically configured to:

In conjunction with the third possible implementation of the second aspect, in a fourth possible implementation, the first processing module is specifically configured to:

With reference to the second aspect to the fourth possible implementation of the second aspect, in a fifth possible implementation, the second processing module is specifically configured to:

Transmitting the encoded instant video frame.

With reference to the fifth possible implementation of the second aspect, in a sixth possible implementation, the second processing module is specifically configured to:

With reference to the second aspect to the sixth possible implementation of the second aspect, in a seventh possible implementation, the third processing module is specifically configured to:

Transmitting the residual data.

With reference to the seventh possible implementation of the second aspect, in the eighth possible implementation, the third processing module is specifically configured to:

Obtaining a position parameter and a posture parameter of the interest area;

Normalizing the position parameter and the posture parameter to generate a standard position parameter and a standard posture parameter;

With reference to the second aspect, in a ninth possible implementation manner, the fourth processing module is specifically configured to:

Obtaining a second feature parameter for describing the region of interest;

Transmitting the second characteristic parameter.

With reference to the ninth possible implementation of the second aspect, in a tenth possible implementation, the fourth processing module is specifically configured to:

With reference to the tenth possible implementation of the second aspect, in the eleventh possible implementation, the fourth processing module is specifically configured to:

In a third aspect, an electronic device is provided, the electronic device comprising a transmitting module, a memory, and the transmitting module, the memory connecting processor, wherein the memory is configured to store a set of program codes, the processing The program code stored by the memory is used to perform the following operations:

Obtaining an area of interest in the current instant video frame;

In conjunction with the third aspect, in a first possible implementation, the processor invokes program code stored in the memory to perform the following operations:

Obtaining a first feature parameter for indicating the region of interest;

In combination with the third aspect or the first possible implementation manner of the third aspect, in a second possible implementation manner, the processor invokes the program code stored by the memory to perform the following operations:

Transmitting the processed instant video frame.

In conjunction with the second possible implementation of the third aspect, in a third possible implementation, the processor invokes the program code stored in the memory to perform the following operations:

In conjunction with the third possible implementation of the third aspect, in a fourth possible implementation, the processor invokes the program code stored in the memory to perform the following operations:

With reference to any of the third aspect to the fourth possible implementation of the third aspect, in a fifth possible implementation, the processor invokes the program code stored in the memory to perform the following operations:

Transmitting the encoded instant video frame.

In conjunction with the fifth possible implementation of the third aspect, in a sixth possible implementation, the processor invokes the program code stored in the memory to perform the following operations:

With reference to the third aspect to the sixth possible implementation of the third aspect, in a seventh possible implementation, the processor invokes the program code stored in the memory to perform the following operations:

Transmitting the residual data.

In conjunction with the seventh possible implementation of the first aspect, in an eighth possible implementation, the processor invokes the program code stored in the memory to perform the following operations:

Obtaining a position parameter and a posture parameter of the interest area;

In conjunction with the first aspect, in a ninth possible implementation, the processor invokes program code stored in the memory to perform the following operations:

Obtaining a second feature parameter for describing the region of interest;

Transmitting the second characteristic parameter.

In conjunction with the ninth possible implementation of the third aspect, in a tenth possible implementation, the processor invokes the program code stored in the memory to perform the following operations:

Generating the second special according to the attitude parameter and the position parameter corresponding to the at least one triangular region Sign the parameters.

In conjunction with the tenth possible implementation of the third aspect, in an eleventh possible implementation, the processor invokes the program code stored in the memory to perform the following operations:

The present invention provides an instant video encoding method and an electronic device, including: acquiring an interest region in a current instant video frame; performing the following operations according to a preset condition that the network bandwidth satisfies and a transmission policy supported by the electronic device; Any one of the following: if the network bandwidth meets the first preset condition, transmitting the current instant video frame including the interest area according to the first transmission policy; if the network bandwidth meets the second preset condition, transmitting, including the interest according to the second transmission policy The current instant video frame of the area; if the network bandwidth meets the third preset condition, transmitting the current instant video frame including the interest area according to the third transmission policy; and if the network bandwidth satisfies the fourth preset condition, according to the fourth transmission policy The transmission includes the current live video frame of the region of interest. By determining whether the network bandwidth meets any one of the first preset condition, the second preset condition, the third preset condition, and the fourth preset condition, different transmission strategies may be adopted for the live video according to different network bandwidths. Compared with the traditional method for transmitting a real-time video, the method improves the utilization of the network bandwidth and saves the network resources, thereby improving the transmission efficiency of the instant video and improving the user experience. In addition, as described in the embodiment of the present invention, The method is: after obtaining the region of interest from the current instant video frame, processing the current instant video frame, so that more bandwidth resources are used in the case that the bandwidth resources are unchanged compared to the conventional real-time video encoding method. The transmission of the interest area ensures the transmission of the interest area, further improves the transmission efficiency of the instant video and improves the user experience.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following description will be made on the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in the drawings The drawings obtain other figures.

FIG. 1 is a flowchart of a method for transmitting instant video according to an embodiment of the present invention;

2 is a flowchart of a method for transmitting an instant video according to an embodiment of the present invention;

3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. Some embodiments of the invention, rather than all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

An embodiment of the present invention provides an instant video encoding method, where the method is used in an interactive system including at least two electronic devices, wherein in the interactive system, the at least two electronic devices are stored by running the self. The application implements real-time video interaction, and the electronic device includes at least a transmitting/receiving module, a video input module, and a display module having a touch function, the video input module includes a camera, the display module may include a display screen, and the electronic device passes the transmitting/receiving module Realizing the sending and receiving of instant video, realizing the input of instant video through the video input module, realizing the display of the instant video and obtaining the instruction triggered by the user through the display module with the touch function.

It is to be noted that the above-mentioned interactive system is merely exemplary, and is used to further illustrate the method provided by the embodiment of the present invention. The specific embodiment of the present invention does not limit the specific interactive system.

Embodiment 1

An embodiment of the present invention provides a method for transmitting an instant video. Referring to FIG. 1, the method includes:

101. Acquire an area of interest in the current instant video frame.

Specifically, acquiring a first feature parameter for indicating the area of interest;

102. Determine whether the network bandwidth meets any one of a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition; if the network bandwidth meets the first preset condition, and the electronic device supports the first a transmission policy, step 103 is performed; if the network bandwidth meets the second preset condition, and the electronic device supports the second transmission policy, step 104 is performed; if the network bandwidth meets the third preset condition, and the electronic device supports the third transmission If the network bandwidth meets the fourth preset condition and the electronic device supports the fourth transmission policy, step 106 is performed.

103. According to the first transmission policy, transmit a current instant video frame including the area of interest, and end.

Specifically, the current live video is processed according to an importance matrix corresponding to the current instant video frame, and the processed instant video frame is obtained;

Transmit processed instant video frames.

If the network bandwidth meets the first preset condition, and the electronic device does not support the first transmission policy, the process ends, and after the end, the instant video frame is transmitted by using a conventional transmission mode.

104. According to the second transmission policy, transmit the current instant video frame including the interest area, and end.

Specifically, the instant video frame including at least the region of interest is divided into at least one macroblock, and the quantization parameter corresponding to the at least one macroblock is set;

Encoding at least one macroblock according to the quantization parameter to generate an encoded instant video frame;

Transmit the encoded instant video frame.

If the network bandwidth meets the second preset condition, and the electronic device supports the second transmission policy, the process ends, and after the end, the instant video frame is transmitted by using a conventional transmission mode.

105. According to the third transmission policy, transmit the current instant video frame including the interest area, and end.

Specifically, acquiring residual data between the current instant video frame and the reference frame including the region of interest;

Transmit residual data.

If the network bandwidth meets the third preset condition, and the electronic device does not support the third transmission policy, the process ends. And after the end, the instant video frame is transmitted by conventional transmission.

106. According to the fourth transmission policy, transmit a current instant video frame including the interest area, and end.

Specifically, acquiring feature parameters for describing an area of interest;

Transfer feature parameters.

If the network bandwidth satisfies the fourth preset condition, and the electronic device does not support the fourth transmission policy, the process ends, and after the end, the instant video frame is transmitted through the conventional transmission mode.

The present invention provides a method for transmitting an instant video, which can be determined according to whether the network bandwidth satisfies any one of a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition. Low network bandwidth adopts different transmission strategies for real-time video. Compared with traditional instant video transmission methods, this method improves network bandwidth utilization and saves network resources, thereby improving the efficiency of real-time video transmission and improving users. In addition, the method according to the embodiment of the present invention processes the current instant video frame after acquiring the region of interest from the current instant video frame, so that the bandwidth resource is unchanged compared to the conventional instant video encoding method. In the case, more broadband resources are used for the transmission of the interest area, thereby ensuring the transmission of the interest area, further improving the transmission efficiency of the instant video and improving the user experience.

Embodiment 2

An embodiment of the present invention provides a method for transmitting an instant video. Referring to FIG. 2, the method includes:

201. Acquire an area of interest in the current instant video frame.

Specifically, the area of interest may include an area where the user details are located, the user details include a face; the area of interest may also include an area where the screen included in the current instant video frame is located, and the area of interest may further include other areas. In the embodiment, the specific region of interest is not limited in the embodiment of the present invention.

For convenience of description, the embodiment of the present invention takes the region where the region of interest is the face, for example, the process may be:

a. acquiring a first feature parameter for indicating an area of interest;

The feature parameters for indicating the area where the face is located include pixels for describing the face and bits of the feature point. Set parameters, etc., which are used to describe the outline of a human face.

b. Acquire an area of interest from the current instant video frame according to the first feature parameter.

Obtaining parameters of all pixels in the region indicated by the feature parameter, the parameters of the pixel include a position parameter of the pixel point, and the position parameter is used to uniquely determine the pixel point.

In the scenario of the instant video interaction, the interaction between the two parties in the instant video, such as the area where the face is located, and the image quality are greater than the other regions. Therefore, the method of the present invention can be used to Perform undistorted transmission, perform corresponding low-pass filtering on the rest, remove high-frequency signals, and reduce the bandwidth occupied by other areas during transmission, so that the bandwidth required by the method is compared to the conventional instant video transmission method. Fewer resources, which can further save bandwidth resources, improve the efficiency of instant video transmission, and improve the user experience.

202. Determine whether the network bandwidth meets any one of a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition; if the network bandwidth meets the first preset condition, and the electronic device supports the first a transmission policy, step 203 is performed; if the network bandwidth meets the second preset condition, and the electronic device supports the second transmission policy, step 205 is performed; if the network bandwidth meets the third preset condition, and the electronic device supports the third transmission If the network bandwidth meets the fourth preset condition and the electronic device supports the fourth transmission policy, step 210 is performed.

Specifically, determining a numerical relationship between the network bandwidth and the first preset value, the second preset value, and the third preset value, the specific manner of determining the specific manner in the embodiment of the present invention is not limited.

If the network bandwidth is greater than or equal to the first preset value, determining that the network bandwidth meets the first preset condition;

If the network bandwidth is less than the first preset value and greater than or equal to the second preset value, determining that the network bandwidth meets the second preset condition;

If the network bandwidth is less than the second preset value and greater than or equal to the third preset value, determining that the network bandwidth meets the third preset condition;

If the network bandwidth is less than the third preset condition, determining that the network bandwidth meets the fourth preset condition.

The process of determining the numerical relationship between the network bandwidth and the first preset value, the second preset value, and the third preset value may be:

Determining whether the network bandwidth is greater than or equal to the first preset value, and if yes, determining that the network bandwidth meets the first preset condition, and performing step 203; if not, determining whether the network bandwidth is greater than or equal to the second preset value, and if so, Then, it is determined that the network bandwidth meets the second preset condition, and step 205 is performed; if not, it is determined whether the network bandwidth is greater than or equal to the third preset value, and if yes, determining that the network bandwidth meets the third preset condition, and performing step 208 If not, it is determined that the network bandwidth meets the fourth preset condition, and step 210 is performed.

The actual preset value may be 100 kb/s, the second preset value may be 50 kb/s, and the third preset value may be 10 kb/s.

Optionally, because the network bandwidth is unstable during network transmission, it may be determined whether the average value of the network bandwidth in the preset time period before the current time meets the first preset condition, the second preset condition, and the third pre-predetermined Set any of the conditions and the fourth preset condition.

Among them, in practical applications, the preset time period may be 0.2 seconds.

By determining whether the average value of the network bandwidth in the preset time period meets the preset condition, the network bandwidth instability during network transmission is eliminated, the reliability of the instant video coding is ensured, and the user experience is improved.

It should be noted that if the network bandwidth is greater than or equal to the first preset value, and the electronic device does not support the first transmission policy, the instant video frame is transmitted through the conventional transmission mode.

If the network bandwidth is less than the first preset value, greater than or equal to the second preset value, and the electronic device does not support the second transmission policy, the instant video frame is transmitted by using a conventional transmission manner.

If the network bandwidth is less than the second preset value, greater than or equal to the third preset value, and the electronic device does not support the third transmission policy, the instant video frame is transmitted through the conventional transmission mode.

If the network bandwidth is less than the third preset value, and the electronic device does not support the fourth transmission policy, the instant video frame is transmitted through the conventional transmission mode.

By determining whether the network bandwidth meets the preset condition, when the network bandwidth is determined to be low, the processing of the instant video can improve the utilization of the network bandwidth, save the network resources, improve the transmission efficiency of the instant video, and improve the user experience.

203. Process the current instant video according to an importance matrix corresponding to the current instant video frame, and obtain the processed instant video frame.

Specifically, according to the region of interest, an importance matrix corresponding to the current instant video frame is generated; the process may be:

Set the importance metric of all pixels in the current instant video frame;

An importance matrix corresponding to the current instant video frame is generated according to the importance metric of all pixels in the current instant video frame after processing.

According to the importance matrix, the current instant video frame is processed, the processed instant video frame is obtained, and the processed instant video frame is input into the video encoder.

The process of low-pass filtering the current instant video frame in the low-pass filtering of the region of interest is the same, and will not be described here.

The processed instant video frame is obtained according to the filtered current instant video frame, the current instant video frame, and the importance matrix.

The current instant video frame and the filtered current instant video frame may be merged on the basis of the importance matrix by using a preset generation algorithm to obtain the processed instant video frame. The algorithm may be:

Where out is the processed instant video frame, src is the current instant video frame, and low is the filtered current instant video frame.

For the importance matrix.

By performing low-pass filtering on the current live video, the high-frequency signal in the current instant video is eliminated, and the bandwidth resource occupied by the current instant video is reduced, thereby further improving the transmission efficiency of the instant video and improving the user experience; The matrix indicates the region of interest and the region of non-interest in the instant video frame. Therefore, the region of interest indicated by the importance matrix is reserved by the importance matrix, and the non-interest region in the low-pass filtered instant video frame is used to replace the instant region. a non-interest area in the video frame, such that the image of the region of interest in the processed processed instant video frame is an undistorted original image, and the image of the non-interest region is an image that does not include a high-frequency signal, so that after the processing is transmitted Instant When the video frame is used, the bandwidth resources required for transmission are reduced, the transmission efficiency of the instant video is improved, and the user experience is improved.

204. The instant video frame after the transmission process ends.

Specifically, the processed instant video frame is input to the encoder, and the processed instant video frame is encoded by the encoder to generate code stream data;

The specific transmission mode is not limited in the embodiment of the present invention.

It should be noted that, in step 203 to step 204, a process of transmitting a current instant video frame including an area of interest according to the first transmission policy is implemented, and the process may be implemented in other manners in addition to the manner described in the foregoing steps. The specific embodiments of the invention are not limited.

The embodiment of the present invention processes the current instant video frame according to the importance matrix, and the importance matrix indicates the importance metric value of each pixel in the current instant video, thereby processing each pixel point, thereby improving the processing. The picture quality of the instant video further enhances the user experience.

205. Divide an instant video frame including at least an area of interest into at least one macroblock, and set a quantization parameter corresponding to the at least one macroblock.

Specifically, the processed current instant video frame may be divided into at least one macroblock according to a preset division rule, and the specific division rule is not limited in the embodiment of the present invention.

Setting the quantization parameter of the macroblock including all or part of the interest area to the first value; since the value of the quantization parameter ranges from 0 to 51 in practical applications, the first value may be between 0 and 25 Or any value defined by any of the systems.

The quantization parameter of the macroblock not including all or part of the region of interest is set to a second value; the second value may be any value defined by any one of the user or system between 25 and 51.

In the scenario of real-time video interaction, the interaction between the two parties in the instant video, such as the area where the face is located, and the image quality are greater than other areas, so the package can be set. The quantization parameter of the macroblock including all or part of the region of interest is the first value. During the encoding process, more details of the region of interest are reserved, so that the image quality of the region of interest can be ensured in the process of instant video, and the user is interested in The requirements of the detailed picture of the area further enhance the user experience. In addition, in the scenario of real-time video interaction, the mutual attention and the picture quality of the interest area in the instant video such as the area where the face is located are greater than other areas, so the macro block not including all or part of the interest area may be set. The quantization parameter is a second value. During the encoding process, the details of the macroblock that does not include all or part of the region of interest are not reserved, and the bandwidth occupied by the macroblocks that do not include all or part of the region of interest is reduced. Compared with the traditional instant video transmission method, the method requires less bandwidth resources, thereby further saving bandwidth resources, improving the efficiency of instant video transmission, and improving the user experience. In addition, by setting the quantization parameter of the macroblock including all or part of the region of interest to the first value, the quantization parameter of the macroblock not including all or part of the region of interest is set to the second value, so that the region of interest is reserved during the encoding process. The details of the non-interest area are not preserved, so that the bandwidth occupied by the non-interest area during transmission is further reduced on the basis of undistorted transmission of the interest area, so that compared with the traditional instant video transmission method The method can not only ensure the picture quality of the region of interest, but also requires less bandwidth resources, thereby further saving bandwidth resources, improving the efficiency of instant video transmission, and improving the user experience.

206. Encode at least one macroblock according to the quantization parameter to generate an encoded instant video frame.

Specifically, the processed instant video frame is encoded according to the first value and the second value.

The specific coding manner is not limited in the embodiment of the present invention.

207. Transmit the encoded instant video frame and end.

Specifically, the specific transmission mode is not limited in the embodiment of the present invention.

It should be noted that, in step 205 to step 207, a process of transmitting a current instant video frame including an area of interest according to the second transmission policy is implemented, and the process may be implemented according to other manners, in addition to the manner described in the foregoing steps. The specific embodiments of the invention are not limited.

Since at least one macroblock is encoded according to a quantization parameter for indicating the degree of fineness of the macroblock in the encoding process, the details of the region of interest are retained during the encoding process, and the non-interest region is fine The section is not reserved, not only uses more broadband resources for the transmission of the interest area, but also ensures the picture quality of the interest area in the instant video, thereby ensuring the transmission of the interest area, further improving the transmission efficiency of the instant video, and improving the efficiency. user experience.

If the network bandwidth meets the second preset condition, and the electronic device does not support the second transmission policy, the process ends, and after the end, the instant video frame is transmitted through the conventional transmission mode.

Optionally, in the actual application, before step 205 is performed, step 203 may also be performed.

208. Acquire residual data between the current instant video frame and the reference frame including the region of interest.

Specifically, the location parameter and the pose parameter of the region of interest are obtained from the current instant video frame.

The position parameter and the attitude parameter are normalized to generate a standard position parameter and a standard posture parameter.

Obtaining residual data between the current instant video frame and the reference frame including the region of interest according to the standard position parameter and the standard posture parameter. The attitude parameter of the region of interest in the reference frame is a standard gesture parameter.

Since the attitude parameter of the region of interest in the current video frame and the gesture parameter of the region of interest in the reference frame are standard gesture parameters, the current location video frame and the reference frame including the region of interest are obtained according to the standard location parameter and the standard gesture parameter. The residual data generates residual data compared to the position parameter and the position parameter in the reference frame according to the posture parameter position parameter of the interest region in the current instant video frame, thereby reducing the data transmission amount and the processing amount in the instant video transmission process. It further improves the transmission efficiency of instant video and improves the user experience. In addition, by acquiring residual data between the interest area in the current instant video frame and the interest area in the reference frame, real-time video transmission is realized, and the method further reduces the instantaneous video transmission compared to the conventional instant video transmission method. The amount of data transmission and processing in the process further provides the transmission efficiency of the instant video and improves the user experience.

209. Transmit the residual data and end.

It should be noted that, in step 208 to step 209, a process of transmitting a current instant video frame including an area of interest according to the third transmission policy is implemented, and in addition to the manner described in the foregoing steps, The specific manner is not limited in the embodiment of the present invention.

Since the residual data between the current instant video frame and the reference frame including the region of interest is acquired and transmitted, the method further saves network bandwidth resources compared to the conventional transmission of all the instant video frames, and ensures the instant video. The picture quality of the area of interest ensures the transmission of the area of interest, thereby further improving the transmission efficiency of the instant video and improving the user experience.

If the network bandwidth satisfies the third preset condition, and the electronic device does not support the third transmission policy, the process ends, and after the end, the instant video frame is transmitted by the conventional transmission mode.

Optionally, in an actual application, at least one of step 205 and step 203 may also be performed before step 208 is performed.

210. Acquire a second feature parameter for describing an area of interest.

Specifically, the region of interest is triangulated to obtain at least one triangular region.

Obtain positional parameters and attitude parameters for describing three feature points of any one of the triangular regions.

According to the position parameter and the attitude parameter of the three feature points, the attitude parameter and the position parameter corresponding to any one of the triangular regions are generated.

Generating a second feature parameter according to the attitude parameter and the position parameter corresponding to the at least one triangular region.

The specific generation manner of the embodiment of the present invention is not limited.

Since the attitude parameter and the position parameter of the triangular area indicate the current position and posture of the triangular area, by adjusting the characteristic parameter, driving of the triangular area can be realized, and the posture of at least one triangular area constituting the interest area in the current instant video is adopted. The parameter and the position parameter are used to adjust the attitude parameter and the position parameter of the same triangular area in the transmitted instant video frame, so as to migrate the interest area in the current instant video frame to the already transmitted instant video frame, compared to transmitting the current live video. The frame further saves network bandwidth, improves the transmission efficiency of instant video, and improves the user experience.

211. Transmit the second feature parameter and end.

It should be noted that, in step 210 to step 211, a process of transmitting a current instant video frame including an area of interest according to the fourth transmission policy is implemented, and in addition to the manner described in the foregoing steps, The specific manner is not limited in the embodiment of the present invention.

By transmitting only the feature parameters for describing the region of interest, the bandwidth resource is further saved compared to transmitting all the instant video frame data, thereby further improving the transmission efficiency of the instant video and improving the user experience.

The present invention provides a method for transmitting an instant video, which can be determined according to whether the network bandwidth satisfies any one of a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition. Low network bandwidth adopts different transmission strategies for real-time video. Compared with traditional instant video transmission methods, this method improves network bandwidth utilization and saves network resources, thereby improving the efficiency of real-time video transmission and improving users. In addition, the method according to the embodiment of the present invention processes the current instant video frame after acquiring the region of interest from the current instant video frame, so that the bandwidth resource is unchanged compared to the conventional instant video encoding method. In the case, more broadband resources are used for the transmission of the interest area, thereby ensuring the transmission of the interest area, further improving the transmission efficiency of the instant video and improving the user experience. In addition, the embodiment of the present invention processes the current instant video frame according to the importance matrix, and the importance matrix indicates the importance metric value of each pixel in the current instant video, thereby processing each pixel point, thereby improving The picture quality of the processed instant video further enhances the user experience. In addition, since at least one macroblock is encoded according to a quantization parameter for indicating the degree of fineness of the macroblock in the encoding process, the details of the region of interest are preserved during the encoding process, and the details of the non-interest region are not retained. Not only the more broadband resources are used for the transmission of the interest area, but also the picture quality of the interest area in the instant video is ensured, thereby ensuring the transmission of the interest area, further improving the transmission efficiency of the instant video and improving the user experience. In addition, since the residual data between the current instant video frame and the reference frame including the region of interest is acquired and transmitted, the method further saves network bandwidth resources and ensures instantness compared to conventional transmission of all real-time video frames. The picture quality of the interest area in the video ensures the transmission of the interest area, thereby further improving the transmission efficiency of the instant video and improving the user experience. In addition, by transmitting only The feature parameters used to describe the interest area further save bandwidth resources compared to transmitting all the real-time video frame data, thereby further improving the transmission efficiency of the instant video and improving the user experience.

Embodiment 3

An embodiment of the present invention provides an electronic device 3. Referring to FIG. 3, the electronic device 3 includes:

The obtaining module 31 is configured to acquire an area of interest in the current instant video frame.

The determining module 32 is configured to determine whether the network bandwidth meets any one of the first preset condition, the second preset condition, the third preset condition, and the fourth preset condition;

The first processing module 33 is configured to: when the network bandwidth meets the first preset condition, transmit the current instant video frame including the interest area according to the first transmission policy; or

The second processing module 34 is configured to: when the network bandwidth meets the second preset condition, transmit the current instant video frame including the interest area according to the second transmission policy; or

The third processing module 35 is configured to: when the network bandwidth meets the third preset condition, transmit the current instant video frame including the interest area according to the third transmission policy; or

The fourth processing module 36 is configured to: when the network bandwidth meets the fourth preset condition, transmit the current instant video frame including the interest area according to the fourth transmission policy.

Optionally, the obtaining module 31 is specifically configured to:

Obtaining a first feature parameter for indicating an area of interest;

Obtaining an area of interest from the current instant video frame according to the first feature parameter.

Optionally, the first processing module 33 is specifically configured to:

The current instant video is processed according to an importance matrix corresponding to the current instant video frame, and the processed instant video frame is obtained;

Transmit processed instant video frames.

Optionally, the first processing module 33 is specifically configured to:

Set the importance metric of all pixels in the current instant video frame;

Optionally, the second processing module 34 is specifically configured to:

Dividing an instant video frame including at least an area of interest into at least one macroblock, and setting a quantization parameter corresponding to the at least one macroblock;

Transmit the encoded instant video frame.

Optionally, the second processing module 34 is specifically configured to:

Setting a quantization parameter of a macroblock including all or part of the region of interest to a first value;

Optionally, the third processing module 35 is specifically configured to:

Obtaining residual data between the current instant video frame including the region of interest and the reference frame;

Transmit residual data.

Optionally, the third processing module 35 is specifically configured to:

Obtaining a position parameter and a posture parameter of the region of interest;

Normalizing the position parameter and the attitude parameter to generate a standard position parameter and a standard posture parameter;

Obtaining residual data between the current instant video frame and the reference frame including the region of interest according to the standard position parameter and the standard posture parameter.

Optionally, the fourth processing module 36 is specifically configured to:

Obtaining feature parameters for describing an area of interest;

Transfer feature parameters.

Optionally, the fourth processing module 36 is specifically configured to:

Triangulating the region of interest to obtain at least one triangular region;

Obtaining at least one posture parameter and a position parameter corresponding to the triangular area;

Optionally, the fourth processing module 36 is specifically configured to:

Generating an attitude parameter and a position parameter corresponding to any one of the triangular regions according to the position parameter and the attitude parameter of the three feature points;

And so on, the at least one triangle region corresponding to the attitude parameter and the position parameter are obtained.

The present invention provides an electronic device, which can determine whether the network bandwidth satisfies any one of a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition. Low network bandwidth adopts different transmission strategies for real-time video. Compared with traditional instant video transmission methods, this method improves network bandwidth utilization and saves network resources, thereby improving the efficiency of real-time video transmission and improving users. In addition, the method according to the embodiment of the present invention processes the current instant video frame after acquiring the region of interest from the current instant video frame, so that the bandwidth resource is unchanged compared to the conventional instant video encoding method. In the case, more broadband resources are used for the transmission of the interest area, thereby ensuring the transmission of the interest area, further improving the transmission efficiency of the instant video and improving the user experience.

Embodiment 4

An embodiment of the present invention provides an electronic device 4. Referring to FIG. 4, the electronic device 5 includes a sending module 41, a memory 42 and a sending module 41. The memory 42 is connected to the processor 43, wherein the memory 42 is used to store a The group program code, the processor 43 calls the program code stored in the memory 42 for performing the following operations:

Obtaining an area of interest in the current instant video frame;

If the network bandwidth meets the fourth preset condition, the current instant video frame including the region of interest is transmitted according to the fourth transmission policy.

Optionally, the processor 43 calls the program code stored in the memory 42 to perform the following operations:

Obtaining a first feature parameter for indicating an area of interest;

Transmit processed instant video frames.

Set the importance metric of all pixels in the current instant video frame;

Transmit the encoded instant video frame.

Transmit residual data.

Obtaining a second feature parameter for describing an area of interest;

Transmitting the second feature parameter.

Triangulating the region of interest to obtain at least one triangular region;

All of the above optional technical solutions may be used in any combination to form an optional embodiment of the present invention, and will not be further described herein.

It is to be noted that the “first”, “second”, “third” and “fourth” in the above embodiments are merely exemplary and not specifically indicated.

It should be noted that, when the electronic device provided by the foregoing embodiment performs the transmission of the instant video, only the division of the above functional modules is illustrated. In actual applications, the function distribution may be completed by different functional modules as needed. The internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the embodiment of the present invention is the same as the embodiment of the method for transmitting an instant video. The specific implementation process is described in detail in the method embodiment, and details are not described herein again.

A person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium. The storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, improvements, etc., which are within the spirit and scope of the present invention, should be included in the protection of the present invention. Within the scope.

Claims

A method for transmitting an instant video, characterized in that the method comprises:

Obtaining an area of interest in the current instant video frame;

Perform any of the following operations based on the preset conditions that the network bandwidth meets and the transmission policies supported by the electronic device:

If the network bandwidth meets the first preset condition, transmitting the current instant video frame including the interest area according to the first transmission policy; or

If the network bandwidth meets the second preset condition, transmitting the current instant video frame including the interest area according to the second transmission policy; or

If the network bandwidth meets the third preset condition, transmitting the current instant video frame including the interest area according to the third transmission policy; or

If the network bandwidth meets the fourth preset condition, transmitting the current instant video frame including the interest area according to the fourth transmission policy.
The method according to claim 1, wherein the obtaining the region of interest in the current instant video frame comprises:

Obtaining a first feature parameter for indicating the region of interest;

Obtaining the region of interest from the current instant video frame according to the first feature parameter.
The method according to claim 1 or 2, wherein the transmitting the current instant video frame including the region of interest according to the first transmission policy comprises:

And processing the current instant video according to an importance matrix corresponding to the current instant video frame, and acquiring the processed instant video frame;

Transmitting the processed instant video frame.
The method according to claim 3, wherein the processing the current instant video according to the importance matrix corresponding to the current instant video frame, and acquiring the processed instant video frame comprises:

Generating an importance matrix corresponding to the current instant video frame according to the region of interest;

And processing, according to the importance matrix, the current instant video frame, acquiring the processed instant video frame, and inputting the processed instant video frame into the video encoder.
The method according to claim 4, wherein the generating an importance matrix corresponding to the current instant video frame according to the region of interest comprises:

Setting an importance metric value of all pixels in the current instant video frame;

Performing low-pass filtering processing on the region of interest to generate a processed current instant video frame;

Generating the importance matrix corresponding to the current instant video frame according to the importance metric of all the pixels in the current current video frame.
The method according to any one of claims 1 to 5, wherein the transmitting the current instant video frame policy including the region of interest according to the second transmission policy comprises:

Dividing an instant video frame including at least the region of interest into at least one macroblock, and setting a quantization parameter corresponding to the at least one macroblock;

Encoding the at least one macroblock according to the quantization parameter to generate an encoded instant video frame;

Transmitting the encoded instant video frame.
The method according to claim 6, wherein the setting the quantization parameter corresponding to the at least one macroblock comprises:

Dividing the processed current instant video frame into at least one macroblock;

Setting a quantization parameter of a macroblock including all or part of the region of interest as a first value;

A quantization parameter that sets a macroblock that does not include all or part of the region of interest is a second value.
The method according to any one of claims 1 to 7, wherein the transmitting the current instant video frame including the region of interest according to the third transmission policy comprises:

Obtaining residual data between the current instant video frame and the reference frame including the region of interest;

Transmitting the residual data.
The method of claim 8 wherein said obtaining comprises said region of interest The residual data between the current instant video frame and the reference frame includes:

Obtaining a position parameter and a posture parameter of the interest area;

Normalizing the position parameter and the posture parameter to generate a standard position parameter and a standard posture parameter;

Obtaining residual data between the current instant video frame and the reference frame including the region of interest according to the standard location parameter and the standard gesture parameter.
The method according to claim 1, wherein the transmitting the current instant video frame including the region of interest according to the fourth transmission policy comprises:

Obtaining a second feature parameter for describing the region of interest;

Transmitting the second characteristic parameter.
The method according to claim 10, wherein the acquiring the second feature parameter for describing the region of interest comprises:

Performing triangulation on the region of interest to acquire at least one triangular region;

Acquiring the attitude parameter and the position parameter corresponding to the at least one triangular area;

Generating the second feature parameter according to the attitude parameter and the position parameter corresponding to the at least one triangular region.
The method according to claim 11, wherein the acquiring the attitude parameter and the position parameter of the at least one triangular region comprises:

Obtaining position parameters and attitude parameters for describing three feature points of any one of the triangular regions;

Generating a posture parameter and a position parameter corresponding to the arbitrary one of the triangular regions according to the position parameter and the posture parameter of the three feature points;

And so on, acquiring the attitude parameter and the position parameter corresponding to the at least one triangular region.
An electronic device, comprising:

Obtaining a module, acquiring an area of interest in the current instant video frame;

a determining module, configured to determine whether the network bandwidth meets any one of a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition;

The decision module is further configured to trigger any one of the following modules according to a preset condition that the network bandwidth meets and a transmission policy supported by the electronic device:

a first processing module, configured to: when the network bandwidth meets the first preset condition, transmit the current instant video frame including the interest area according to the first transmission policy; or

a second processing module, configured to: when the network bandwidth meets a second preset condition, transmit the current instant video frame that includes the interest area according to a second transmission policy; or

a third processing module, configured to: when the network bandwidth meets a third preset condition, transmit the current instant video frame including the interest area according to a third transmission policy; or

And a fourth processing module, configured to: when the network bandwidth meets the fourth preset condition, transmit the current instant video frame including the interest area according to the fourth transmission policy.
The device according to claim 13, wherein the first processing module is specifically configured to:

And processing the current instant video according to an importance matrix corresponding to the current instant video frame, and acquiring the processed instant video frame;

Transmitting the processed instant video frame.
The device according to claim 14, wherein the second processing module is specifically configured to:

Dividing an instant video frame including at least the region of interest into at least one macroblock, and setting a quantization parameter corresponding to the at least one macroblock;

Encoding the at least one macroblock according to the quantization parameter to generate an encoded instant video frame;

Transmitting the encoded instant video frame.
The device according to claim 15, wherein the third processing module is specifically configured to:

Obtaining residual data between the current instant video frame and the reference frame including the region of interest;

Transmitting the residual data.
The device according to claim 13, wherein the fourth processing module is specifically configured to:

Obtaining a second feature parameter for describing the region of interest;

Transmitting the second characteristic parameter.