CN111131743A

CN111131743A - Video call method and device based on browser, electronic equipment and storage medium

Info

Publication number: CN111131743A
Application number: CN201911359649.1A
Authority: CN
Inventors: 颜祝俊; 王康桑; 张美聪; 王艳辉
Original assignee: Visionvera Information Technology Co Ltd
Current assignee: Visionvera Information Technology Co Ltd
Priority date: 2019-12-25
Filing date: 2019-12-25
Publication date: 2020-05-08

Abstract

The invention provides a video call method, a device, electronic equipment and a storage medium based on a browser, wherein the method is applied to a video network server in a video call system, the video call system also comprises a mobile terminal and a PC (personal computer) end, and the browser runs on the PC end; the method comprises the following steps: in the video call, receiving first video data sent by a browser, and converting the first video data into second video data conforming to a first preset video coding protocol; receiving third video data sent by the mobile terminal, and converting the third video data into fourth video data conforming to a second preset video coding protocol; sending second video data to the mobile terminal, and sending fourth video data to the browser; the first preset video coding protocol is a protocol adaptive to the mobile terminal, and the second preset video coding protocol is a protocol adaptive to the browser. Therefore, the video call between the browser at the PC end and the mobile terminal is realized.

Description

Video call method and device based on browser, electronic equipment and storage medium

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a video call method and apparatus based on a browser, an electronic device, and a storage medium.

Background

The video networking is an important milestone for network development, is a real-time network, can realize high-definition video real-time transmission, and pushes a plurality of internet applications to high-definition video, and high-definition faces each other.

Along with the expansion of video networking services, more diversified demands are provided for video communication modes based on video networking, and different video communication modes in different modes need to correspond to different video communication mechanisms so as to meet the video communication demands of various pictures of users.

For example, in an application scenario of video communication in the video network, two parties of the video communication need to perform a video call through different devices and different application programs. However, the related art does not support the video call by the user using different devices and different applications by both parties.

Disclosure of Invention

In view of the above, embodiments of the present invention are proposed to provide a browser-based video call method, apparatus, electronic device and storage medium that overcome or at least partially solve the above problems.

In order to solve the above problem, a first aspect of an embodiment of the present invention provides a video call method based on a browser, where the method is applied to a video networking server in a video call system, the video call system further includes a mobile terminal and a PC terminal, and the PC terminal runs on the browser; the method comprises the following steps:

receiving first video data sent by the browser in a video call between the PC end and the mobile terminal, and converting the first video data into second video data conforming to a first preset video coding protocol; receiving third video data sent by the mobile terminal, and converting the third video data into fourth video data conforming to a second preset video coding protocol;

sending the second video data to the mobile terminal, and sending the fourth video data to the browser;

the first preset video coding protocol is a protocol adaptive to the mobile terminal, and the second preset video coding protocol is a protocol adaptive to the browser.

Optionally, converting the first video data into second video data conforming to a first preset video coding protocol includes:

decoding the first video data to obtain first original video data;

according to the first preset video coding protocol, coding the first original video data to obtain second video data;

converting the third video data into fourth video data conforming to a second preset video coding protocol, including:

decoding the third video data to obtain third original video data;

and coding the third original video data according to the second preset video coding protocol to obtain the fourth video data.

Optionally, receiving the first video data sent by the browser includes:

receiving first video data transmitted by the browser based on an RTP protocol or an RTCP protocol;

sending the fourth video data to the browser, including:

and sending the fourth video data to the browser based on the RTP protocol or the RTCP protocol.

Optionally, the video call system further includes a first server and a second server; receiving first video data sent by the browser, and converting the first video data into second video data conforming to a first preset video coding protocol; and receiving third video data sent by the mobile terminal, and converting the third video data into fourth video data conforming to a second preset video coding protocol, including:

receiving first video data sent by the first server and third video data sent by the second server, wherein the first video data are sent to the first server by the browser, and the third video data are sent to the second server by the mobile terminal;

encoding the first video data into first transcoding video data which accords with a second preset video encoding protocol, and adding a video networking protocol packet header to the first transcoding video data to obtain second video data;

encoding the third video data into second transcoded video data conforming to a second preset video encoding protocol, and adding an internet protocol packet header to the second transcoded video data to obtain fourth video data;

sending the second video data to the mobile terminal and sending the fourth video data to the browser, including:

sending the second video data to the second server and sending the fourth video data to the first server; the second server is used for sending the second video data to the mobile terminal, and the first server is used for sending the fourth video data to the browser.

Optionally, the first preset video coding protocol is a VP8 protocol, and the second preset video coding protocol is an h.264 video protocol.

In a second aspect of the embodiments of the present invention, a video call device based on a browser is provided, where the device is located in a video networking server in a video call system, the video call system further includes a mobile terminal and a PC terminal, and the PC terminal runs on the browser; the device comprises:

the video receiving module is used for receiving first video data sent by the browser and third video data sent by the mobile terminal in a video call between the PC end and the mobile terminal;

the video transcoding module is used for converting the first video data into second video data conforming to a first preset video coding protocol and converting the third video data into fourth video data conforming to a second preset video coding protocol;

the video sending module is used for sending the second video data to the mobile terminal and sending the fourth video data to the browser;

Optionally, the video transcoding module includes:

a first decoding unit, configured to decode the first video data to obtain first original video data;

a first encoding unit, configured to encode the first original video data according to the first preset video encoding protocol to obtain the second video data;

a second decoding unit, configured to decode the third video data to obtain third original video data;

and the second encoding unit is configured to encode the third original video data according to the second preset video encoding protocol to obtain the fourth video data.

Optionally, the video receiving module is specifically configured to receive, based on an RTP protocol or an RTCP protocol, first video data sent by the browser;

the video sending module is specifically configured to send the fourth video data to the browser based on the RTP protocol or the RTCP protocol.

In a third aspect of the embodiments of the present invention, there is provided an electronic device, including: one or more processors; and one or more machine readable media having instructions stored thereon, which when executed by the one or more processors, cause the apparatus to perform one or more browser-based video call methods as described in embodiments of the invention.

In a fourth aspect of the embodiments of the present invention, a computer-readable storage medium is provided, which stores a computer program for causing a processor to execute the method for video call based on a browser according to the embodiments of the present invention.

The embodiment of the invention has the following advantages:

in the embodiment of the invention, when the PC and the mobile terminal perform a video call, the PC and the mobile terminal may receive first video data sent by a browser on the PC, receive third video data sent by the mobile terminal, convert the first video data into second video data of a first preset video coding protocol adapted to the mobile terminal, convert the third video data into fourth video data of a second preset video coding protocol adapted to the browser, send the second video data to the mobile terminal, and send the fourth video data to the browser. Therefore, the video coding protocol of the video data is converted by the video networking server, so that the mutual video call between the user of the browser and the user of the mobile terminal is realized, and the diversity of the video call modes of the video networking is expanded.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.

FIG. 1 is a schematic networking diagram of a video network of the present invention;

FIG. 2 is a schematic diagram of a hardware architecture of a node server according to the present invention;

fig. 3 is a schematic diagram of a hardware structure of an access switch of the present invention;

fig. 4 is a schematic diagram of a hardware structure of an ethernet protocol conversion gateway according to the present invention;

fig. 5 is a schematic structural diagram of a video call system according to an embodiment of the present invention;

FIG. 6 is a flowchart illustrating steps of a method for a browser-based video call according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of another video call system according to an embodiment of the present invention;

fig. 8 is a schematic diagram of a framework of a video call device based on a browser according to an embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

The video networking adopts a real-time high-definition video exchange technology, can integrate required services such as dozens of services of video, voice, pictures, characters, communication, data and the like on a system platform on a network platform, such as high-definition video conference, video monitoring, intelligent monitoring analysis, emergency command, digital broadcast television, delayed television, network teaching, live broadcast, VOD on demand, television mail, Personal Video Recorder (PVR), intranet (self-office) channels, intelligent video broadcast control, information distribution and the like, and realizes high-definition quality video broadcast through a television or a computer.

To better understand the embodiments of the present invention, the following description refers to the internet of view:

some of the technologies applied in the video networking are as follows:

network Technology (Network Technology)

Network technology innovation in video networking has improved the traditional Ethernet (Ethernet) to face the potentially huge first video traffic on the network. Unlike pure network Packet Switching (Packet Switching) or network Circuit Switching (Circuit Switching), the Packet Switching is adopted by the technology of the video networking to meet the Streaming requirement. The video networking technology has the advantages of flexibility, simplicity and low price of packet switching, and simultaneously has the quality and safety guarantee of circuit switching, thereby realizing the seamless connection of the whole network switching type virtual circuit and the data format.

Switching Technology (Switching Technology)

The video network adopts two advantages of asynchronism and packet switching of the Ethernet, eliminates the defects of the Ethernet on the premise of full compatibility, has end-to-end seamless connection of the whole network, is directly communicated with a user terminal, and directly bears an IP data packet. The user data does not require any format conversion across the entire network. The video networking is a higher-level form of the Ethernet, is a real-time exchange platform, can realize the real-time transmission of the whole-network large-scale high-definition video which cannot be realized by the existing Internet, and pushes a plurality of network video applications to high-definition and unification.

Server Technology (Server Technology)

The server technology on the video networking and unified video platform is different from the traditional server, the streaming media transmission of the video networking and unified video platform is established on the basis of connection orientation, the data processing capacity of the video networking and unified video platform is independent of flow and communication time, and a single network layer can contain signaling and data transmission. For voice and video services, the complexity of video networking and unified video platform streaming media processing is much simpler than that of data processing, and the efficiency is greatly improved by more than one hundred times compared with that of a traditional server.

Storage Technology (Storage Technology)

The super-high speed storage technology of the unified video platform adopts the most advanced real-time operating system in order to adapt to the media content with super-large capacity and super-large flow, the program information in the server instruction is mapped to the specific hard disk space, the media content is not passed through the server any more, and is directly sent to the user terminal instantly, and the general waiting time of the user is less than 0.2 second. The optimized sector distribution greatly reduces the mechanical motion of the magnetic head track seeking of the hard disk, the resource consumption only accounts for 20% of that of the IP internet of the same grade, but concurrent flow which is 3 times larger than that of the traditional hard disk array is generated, and the comprehensive efficiency is improved by more than 10 times.

Network Security Technology (Network Security Technology)

The structural design of the video network completely eliminates the network security problem troubling the internet structurally by the modes of independent service permission control each time, complete isolation of equipment and user data and the like, generally does not need antivirus programs and firewalls, avoids the attack of hackers and viruses, and provides a structural carefree security network for users.

Service Innovation Technology (Service Innovation Technology)

The unified video platform integrates services and transmission, and is not only automatically connected once whether a single user, a private network user or a network aggregate. The user terminal, the set-top box or the PC are directly connected to the unified video platform to obtain various multimedia video services in various forms. The unified video platform adopts a menu type configuration table mode to replace the traditional complex application programming, can realize complex application by using very few codes, and realizes infinite new service innovation.

Networking of the video network is as follows:

the video network is a centralized control network structure, and the network can be a tree network, a star network, a ring network and the like, but on the basis of the centralized control node, the whole network is controlled by the centralized control node in the network.

As shown in fig. 1, the video network is divided into an access network and a metropolitan network.

The devices of the access network part can be mainly classified into 3 types: node server, access switch, terminal (including various set-top boxes, coding boards, memories, etc.). The node server is connected to an access switch, which may be connected to a plurality of terminals and may be connected to an ethernet network.

The node server is a node which plays a centralized control function in the access network and can control the access switch and the terminal. The node server can be directly connected with the access switch or directly connected with the terminal.

Similarly, devices of the metropolitan network portion may also be classified into 3 types: a metropolitan area server, a node switch and a node server. The metro server is connected to a node switch, which may be connected to a plurality of node servers.

The node server is a node server of the access network part, namely the node server belongs to both the access network part and the metropolitan area network part.

The metropolitan area server is a node which plays a centralized control function in the metropolitan area network and can control a node switch and a node server. The metropolitan area server can be directly connected with the node switch or directly connected with the node server.

Therefore, the whole video network is a network structure with layered centralized control, and the network controlled by the node server and the metropolitan area server can be in various structures such as tree, star and ring.

The access network part can form a unified video platform (the part in the dotted circle), and a plurality of unified video platforms can form a video network; each unified video platform may be interconnected via metropolitan area and wide area video networking.

Video networking device classification

1.1 devices in the video network of the embodiment of the present invention can be mainly classified into 3 types: server, exchanger (including Ethernet protocol conversion gateway), terminal (including various set-top boxes, code board, memory, etc.). The video network as a whole can be divided into a metropolitan area network (or national network, global network, etc.) and an access network.

1.2 wherein the devices of the access network part can be mainly classified into 3 types: node server, access exchanger (including Ethernet protocol conversion gateway), terminal (including various set-top boxes, coding board, memory, etc.).

The specific hardware structure of each access network device is as follows:

a node server:

as shown in fig. 2, the system mainly includes a network interface module 201, a switching engine module 202, a CPU module 203, and a disk array module 204;

the network interface module 201, the CPU module 203, and the disk array module 204 all enter the switching engine module 202; the switching engine module 202 performs an operation of looking up the address table 205 on the incoming packet, thereby obtaining the direction information of the packet; and stores the packet in a queue of the corresponding packet buffer 206 based on the packet's steering information; if the queue of the packet buffer 206 is nearly full, it is discarded; the switching engine module 202 polls all packet buffer queues for forwarding if the following conditions are met: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero. The disk array module 204 mainly implements control over the hard disk, including initialization, read-write, and other operations on the hard disk; the CPU module 203 is mainly responsible for protocol processing with an access switch and a terminal (not shown in the figure), configuring an address table 205 (including a downlink protocol packet address table, an uplink protocol packet address table, and a data packet address table), and configuring the disk array module 204.

The access switch:

as shown in fig. 3, the network interface module mainly includes a network interface module (a downlink network interface module 301 and an uplink network interface module 302), a switching engine module 303 and a CPU module 304;

wherein, the packet (uplink data) coming from the downlink network interface module 301 enters the packet detection module 305; the packet detection module 305 detects whether the Destination Address (DA), the Source Address (SA), the packet type, and the packet length of the packet meet the requirements, and if so, allocates a corresponding stream identifier (stream-id) and enters the switching engine module 303, otherwise, discards the stream identifier; the packet (downstream data) coming from the upstream network interface module 302 enters the switching engine module 303; the incoming data packet of the CPU module 304 enters the switching engine module 303; the switching engine module 303 performs an operation of looking up the address table 306 on the incoming packet, thereby obtaining the direction information of the packet; if the packet entering the switching engine module 303 is from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 307 in association with the stream-id; if the queue of the packet buffer 307 is nearly full, it is discarded; if the packet entering the switching engine module 303 is not from the downlink network interface to the uplink network interface, the data packet is stored in the queue of the corresponding packet buffer 307 according to the guiding information of the packet; if the queue of the packet buffer 307 is nearly full, it is discarded.

The switching engine module 303 polls all packet buffer queues and may include two cases:

if the queue is from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) the port send buffer is not full; 2) the queued packet counter is greater than zero; 3) obtaining a token generated by a code rate control module;

if the queue is not from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero.

The rate control module 308 is configured by the CPU module 304, and generates tokens for packet buffer queues from all downstream network interfaces to upstream network interfaces at programmable intervals to control the rate of upstream forwarding.

The CPU module 304 is mainly responsible for protocol processing with the node server, configuration of the address table 306, and configuration of the code rate control module 308.

Ethernet protocol gateway:

as shown in fig. 4, the apparatus mainly includes a network interface module (a downlink network interface module 401 and an uplink network interface module 402), a switching engine module 403, a CPU module 404, a packet detection module 405, a rate control module 408, an address table 406, a packet buffer 407, a MAC adding module 409, and a MAC deleting module 410.

Wherein, the data packet coming from the downlink network interface module 401 enters the packet detection module 405; the packet detection module 405 detects whether the ethernet MAC DA, the ethernet MAC SA, the ethernet length or frame type, the video network destination address DA, the video network source address SA, the video network packet type, and the packet length of the packet meet the requirements, and if so, allocates a corresponding stream identifier (stream-id); then, the MAC deletion module 410 subtracts MAC DA, MAC SA, length or frame type (2byte) and enters the corresponding receiving buffer, otherwise, discards it;

the downlink network interface module 401 detects the sending buffer of the port, and if there is a packet, obtains the ethernet MAC DA of the corresponding terminal according to the destination address DA of the packet, adds the ethernet MAC DA of the terminal, the MACSA of the ethernet coordination gateway, and the ethernet length or frame type, and sends the packet.

The other modules in the ethernet protocol gateway function similarly to the access switch.

A terminal:

the system mainly comprises a network interface module, a service processing module and a CPU module; for example, the set-top box mainly comprises a network interface module, a video and audio coding and decoding engine module and a CPU module; the coding board mainly comprises a network interface module, a video and audio coding engine module and a CPU module; the memory mainly comprises a network interface module, a CPU module and a disk array module.

1.3 devices of the metropolitan area network part can be mainly classified into 2 types: node server, node exchanger, metropolitan area server. The node switch mainly comprises a network interface module, a switching engine module and a CPU module; the metropolitan area server mainly comprises a network interface module, a switching engine module and a CPU module.

2. Video networking packet definition

2.1 Access network packet definition

The data packet of the access network mainly comprises the following parts: destination Address (DA), Source Address (SA), reserved bytes, payload (pdu), CRC.

As shown in the following table, the data packet of the access network mainly includes the following parts:

DA SA Reserved Payload CRC

wherein:

the Destination Address (DA) is composed of 8 bytes (byte), the first byte represents the type of the data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), there are 256 possibilities at most, the second byte to the sixth byte are metropolitan area network addresses, and the seventh byte and the eighth byte are access network addresses;

the Source Address (SA) is also composed of 8 bytes (byte), defined as the same as the Destination Address (DA);

the reserved byte consists of 2 bytes;

the payload part has different lengths according to different types of datagrams, and is 64 bytes if the datagram is various types of protocol packets, and is 32+1024 or 1056 bytes if the datagram is a unicast packet, of course, the length is not limited to the above 2 types;

the CRC consists of 4 bytes and is calculated in accordance with the standard ethernet CRC algorithm.

2.2 metropolitan area network packet definition

The topology of a metropolitan area network is a graph and there may be 2, or even more than 2, connections between two devices, i.e., there may be more than 2 connections between a node switch and a node server, a node switch and a node switch, and a node switch and a node server. However, the metro network address of the metro network device is unique, and in order to accurately describe the connection relationship between the metro network devices, parameters are introduced in the embodiment of the present invention: a label to uniquely describe a metropolitan area network device.

In this specification, the definition of the Label is similar to that of the Label of MPLS (Multi-Protocol Label Switch), and assuming that there are two connections between the device a and the device B, there are 2 labels for the packet from the device a to the device B, and 2 labels for the packet from the device B to the device a. The label is classified into an incoming label and an outgoing label, and assuming that the label (incoming label) of the packet entering the device a is 0x0000, the label (outgoing label) of the packet leaving the device a may become 0x 0001. The network access process of the metro network is a network access process under centralized control, that is, address allocation and label allocation of the metro network are both dominated by the metro server, and the node switch and the node server are both passively executed, which is different from label allocation of MPLS, and label allocation of MPLS is a result of mutual negotiation between the switch and the server.

As shown in the following table, the data packet of the metro network mainly includes the following parts:

DA SA Reserved Label Payload CRC

Namely Destination Address (DA), Source Address (SA), Reserved byte (Reserved), tag, payload (pdu), CRC. The format of the tag may be defined by reference to the following: the tag is 32 bits with the upper 16 bits reserved and only the lower 16 bits used, and its position is between the reserved bytes and payload of the packet.

Based on the characteristics of the video network, video call modes based on the video network are more and more. For example, in an application scenario, a user needs to perform a video call with a mobile phone terminal through a browser of a PC terminal, in the existing manner, a mobile terminal and the PC terminal can establish a video call connection, but after the video call connection is established, the video call connection does not represent the realization of the video call, and the two video calls are really realized on the basis that the video of the other party can be seen. However, since the PC side performs a video call through the browser, the mobile terminal generally also performs a video call using an application program configured on the mobile terminal, and the coding and decoding protocols adopted by the mobile terminal and the application program may not be the same, which causes the following problems in practice:

on one hand, the browser at the PC end cannot correctly decode the video data sent by the mobile terminal, and the mobile terminal may also not correctly decode the video data sent by the browser, so that the video call in the application scene cannot be really realized.

On the other hand, due to the difference of the encoding and decoding protocols, when the video data sent by the mobile terminal is transmitted to the browser, the video data of the mobile terminal occupies a larger bandwidth, which causes the delay of video call. When the video data sent by the browser is transmitted to the mobile terminal, the mobile terminal may not be clear when decoding and playing the video data, and thus video call cannot be realized.

In view of the above technical problems, the present inventors have proposed a browser-based video call method so that a browser on a PC (personal computer) side can implement a video call with a mobile terminal.

Referring to fig. 5, a schematic structural diagram of a video call system in a video call method based on a browser according to an embodiment of the present application is shown, and as shown in fig. 5, the video call system includes a video networking server, a mobile terminal, and a PC terminal, and the browser is run on the PC terminal, where both the PC terminal and the mobile terminal may be in communication connection with the video networking server.

With reference to the schematic structural diagram of the video call system shown in fig. 5, a detailed description is given of a video call method based on a browser in the embodiment of the present application, and referring to fig. 6, a flowchart of steps of the video call method based on a browser is shown, where the method may be applied to a video networking server, and specifically may include the following steps:

step S601: receiving first video data sent by the browser in a video call between the PC end and the mobile terminal, and converting the first video data into second video data conforming to a first preset video coding protocol; and receiving third video data sent by the mobile terminal, and converting the third video data into fourth video data conforming to a second preset video coding protocol.

In this embodiment, the PC terminal may refer to a computer or a tablet computer, and the mobile terminal may be, but is not limited to, a smart phone. In practice, the PC end and the mobile terminal may establish a video call connection in advance through an SDP (Session Description Protocol), and after establishing the video call connection, the mobile terminal and the PC end may perform a video call based on the established video call connection.

In particular, an application object may be configured on the mobile terminal, which may be used to support a video call conducted on the mobile terminal. For example, it may be a video call APP. In this embodiment, the video networking server may receive the first video data sent by the browser of the PC end and receive the third video data sent by the mobile terminal in the process of the video call between the mobile terminal and the PC end.

The first video data may be a local Web video encoding stream acquired by a browser at the PC end through a WEBRTC (Web Real-time communication) video engine. Among them, WebRTC enables web-based video conferencing. In practice, during the video call of the browser on the PC side, the browser may call the WEBRTC video engine to obtain the video captured by the local camera on the PC side and call the WEBRTC video engine to encode the captured video into the first video data. The third video data may be video data obtained by the mobile terminal after video coding and collected by a camera of the mobile terminal, and the third video data is sent to the video networking server through a configured application program object.

The first video data may be data conforming to a second predetermined video coding protocol, and the third video data may be data conforming to the first predetermined video coding protocol.

After receiving the first video data, the first video data may be converted to second video data that may be received and decoded by the mobile terminal, and specifically, the first video data may be converted to second video data conforming to a first predetermined video coding protocol. Similarly, the third video data may also be converted into fourth video data that can be received and decoded by the browser, and specifically, the third video data may be converted into fourth video data conforming to a second preset video coding protocol.

In this embodiment, the video coding protocol supported by the application object on the mobile terminal may be predetermined, and the video coding protocol supported by the application object is pre-entered into the video networking server as the first preset video coding protocol. The video coding protocol supported by the browser can be pre-determined, and the video coding protocol supported by the browser is pre-recorded into the video networking server to serve as a second preset video coding protocol.

Step S602: and sending the second video data to the mobile terminal, and sending the fourth video data to the browser.

In this embodiment, the video network server may send the second video data to the mobile terminal, and send the fourth video data to the browser. The mobile terminal can decode and play the second video data through the configured target application object, and the browser can play the fourth video data.

In this embodiment, the first preset video coding protocol is a protocol adapted to the mobile terminal, and the second preset video coding protocol is a protocol adapted to the browser, so that the first video data sent by the browser is converted into the second video data adapted to the mobile terminal, the third video data sent by the mobile terminal is converted into the fourth video data adapted to the browser, and then the mobile terminal can decode and play the video sent by the browser, and the browser can decode and play the video sent by the mobile terminal, thereby implementing a real video call between the browser at the PC side and the mobile terminal.

In an alternative embodiment, the first predetermined video coding protocol may be a VP8 protocol, and the second predetermined video coding protocol is an h.264 video protocol. I.e. adapted to the browser, may be the VP8 protocol, which VP8 protocol may be understood to be the video encoding protocol employed by the VP8 video encoder. Which can provide higher quality video with less data and require less processing power to play the video.

When the embodiment is adopted, because the video coding protocol adopted by the second video data is the h.264 protocol adapted to the mobile terminal, when the mobile terminal plays the second video data, the video picture is adapted to the display screen of the mobile terminal, and further the quality of the video picture is improved. And because the video coding protocol of the fourth video data is the VP8 protocol matched with the browser, when the fourth video data is transmitted, the fourth video data can provide high-quality video with less data, so that the occupation of network bandwidth from the video networking server to the browser is reduced, the transmission speed of the video data is increased, the video call delay is further shortened, and the real-time performance of the video call is improved.

With reference to the above embodiments, in an embodiment provided by the present application, converting the first video data into second video data conforming to a first preset video coding protocol in step S601 may include the following steps:

step S6011: and decoding the first video data to obtain first original video data.

The video network server may decode the first video data by using a coding and decoding corresponding to the second preset coding protocol to obtain first original video data, where the first original video data is decoded video data.

Step S6012: and coding the first original video data according to the first preset video coding protocol to obtain the second video data.

After the first original video data is obtained, the first original video data can be coded according to a first preset video coding protocol, and the obtained coded second video data is data which accords with the first preset video coding protocol, so that the mobile terminal can receive and play the data conveniently.

Accordingly, converting the third video data into fourth video data conforming to a second preset video coding protocol may include the steps of:

step S6013: and decoding the third video data to obtain third original video data.

The video network server may decode the third video data by using the codec corresponding to the first preset encoding protocol to obtain third original video data, where the third original video data is the decoded video data.

Step S6014: and coding the third original video data according to the second preset video coding protocol to obtain the fourth video data.

After the third original video data is obtained, the third original video data can be encoded according to a second preset video encoding protocol, and the obtained encoded fourth video data is data conforming to the second preset video encoding protocol, so that the browser can receive and play the data conveniently.

With reference to the foregoing embodiment, in an embodiment provided in this application, the first video data sent by the browser is received in step S601, and the first video data sent by the browser may be received based on an RTP (Real-time Transport Protocol) Protocol or an RTCP (Real-time control Protocol) Protocol.

Accordingly, when the fourth video data is sent to the browser, the fourth video data may also be sent to the browser based on the RTP protocol or the RTCP protocol.

By adopting the embodiment, the video call mode of the application can be carried out by adopting an RTP (real-time transport protocol) or an RTCP (real-time transport control protocol) so as to obtain better video call real-time property.

In an optional embodiment, the third video data sent by the mobile terminal may also be received based on an RTP protocol or the RTCP protocol, and when the second video data is sent to the mobile terminal, the second video data may also be sent to the mobile terminal based on the RTP protocol or the RTCP protocol.

Referring to fig. 7, a schematic structural diagram of a video call system in an embodiment of the present disclosure is shown, and as shown in fig. 7, the video call system may further include a first server and a second server in addition to the video network server, the PC terminal, and the mobile terminal. The PC end can be in communication connection with the first server, the second server can be in communication connection with the mobile terminal, and the first server and the second server are in communication connection with the video network server respectively.

In practice, the first server may be a web server and the second server may be a streaming server. The first server and the PC end can be located in the Internet, and the second server can be located in the video network. Therefore, the PC terminal in the Internet can carry out video call with the mobile terminal in the video network through the browser.

Specifically, in step S601, the following steps may be specifically included:

step S6011': and receiving first video data sent by the first server and receiving third video data sent by the second server.

The first video data are sent to the first server by the browser, and the third video data are sent to the second server by the mobile terminal.

In practice, the first video data sent by the browser may be obtained from the first server by the video network server, and specifically, the browser may send the first video data to the first server, which sends the first video data to the video network server. In practice, the first server may be a web server, which may receive the first video data based on an internet protocol and send it to the video networking server.

Similarly, the video network server may also obtain the third video data sent by the mobile terminal from the second server. The first video data is video data conforming to the internet protocol, and the third video data is video data conforming to the video networking protocol. Wherein, according to the internet protocol, it is understood that the first video data can be transmitted in the internet according to the transmission protocol of the internet. Conforming to the video networking protocol is to be understood as meaning that the third video data can be transmitted in the video networking according to the transmission protocol of the video networking.

Step S6012': and coding the first video data into first transcoding video data conforming to a second preset video coding protocol, and adding a video networking protocol packet header to the first transcoding video data to obtain second video data.

In this embodiment, the first video data is encoded into video data conforming to a second predetermined video encoding protocol. In practice, in order to enable the first transcoded video data to be transmitted in the video network, the video network server may add a video network protocol header to the first transcoded video data, so as to obtain the second video data. Specifically, the video networking protocol header may be understood as an identifier of a video networking transmission protocol added to a specified field of a data packet of the first transcoded video data, and the video networking protocol header may enable a device in the video networking to determine that the second video data is video data transmitted in the video networking, so as to process the second video data.

Step S6013': and encoding the third video data into second transcoding video data which accord with a second preset video encoding protocol, and adding an internet protocol packet header to the second transcoding video data to obtain fourth video data.

In this embodiment, the third transcoded video data is video data obtained by encoding the third video data to conform to the first preset video encoding protocol. In practice, in order to transmit the third transcoded video data to the browser in the internet, the video network server may add an internet protocol header to the third transcoded video data, so as to obtain the fourth video data. Specifically, the internet protocol header may be understood as an identifier of an internet transport protocol added to a specified field of a packet of the third transcoded video data, and the internet protocol header may enable a device in the internet to determine that the fourth video data is video data transmitted in the internet, so as to process the fourth video data.

Correspondingly, the sending of the second video data to the mobile terminal and the sending of the fourth video data to the browser may specifically be as follows:

step S602': sending the second video data to the second server, and sending the fourth video data to the first server.

The second server is used for sending the second video data to the mobile terminal, and the first server is used for sending the fourth video data to the browser.

In this embodiment, the video networking server may send the second video data to the second server, and send the fourth video data to the first server, so that the second server may send the second video data to the mobile terminal based on the video networking protocol, and the first server may send the fourth video data to the browser based on the internet protocol.

When the embodiment is adopted, the mobile terminal in the video network and the PC terminal in the Internet can carry out video call, so that the diversity of video call modes of the video network is further expanded.

In the following, with reference to fig. 7, a video call system may include a web page end (i.e., a PC end running a browser), a mobile phone end, a first server, a second server, and a transcoder. The coding transcoder can run on a video networking server, a mobile application APP can run on the mobile phone terminal, and the mobile application APP can provide social services such as video call for users.

The video call method based on the browser provided by the embodiment of the present application can be implemented by the transcoder, and the following flow description of the video call method based on the browser provided by the embodiment of the present application is as follows, starting with sending a video from the browser side:

firstly, a browser at a webpage end acquires a local webpage video coding stream (namely first video data) by using a video coding and decoding protocol of VP8 through a WEBRTC video engine.

And then, the browser at the webpage end pushes the local webpage video coding stream to a first server through an RTP protocol or an RTCP protocol in a WEBRTC video engine, and the first server pushes the local webpage video coding stream to a coding transcoder in the video network.

And then, the coding transcoder sends the local webpage video coding stream to the decoder for decoding, and after decoding, the decoded video data is coded again according to the H.264 video coding and decoding protocol to obtain video data (namely second video data) adaptive to the mobile phone terminal.

And finally, the coding transcoder can send the video data adaptive to the mobile phone end to a second server through the video networking server, and the second server sends the video data adaptive to the mobile phone end. Therefore, the mobile phone end can play the video data by utilizing the mobile application APP.

Correspondingly, starting from the video transmission from the mobile phone side, the flow of the video call method based on the browser provided by the embodiment of the application is described as follows:

firstly, a mobile application APP at a mobile phone end pushes local mobile phone video data (namely third video data) collected by a mobile phone camera to a second server, and the second server pushes the local mobile phone video data to a coding transcoder in a video network.

And then, the coding transcoder sends the local mobile phone video data to the decoder for decoding, and after decoding, codes the decoded video data again according to the VP8 video coding and decoding protocol to obtain video data (namely fourth video data) adaptive to the webpage end.

Finally, the coding transcoder can send the video data adaptive to the webpage end to the first server through the video networking server, and the first server sends the video data adaptive to the webpage end. Therefore, the webpage end can play the video data on the browser.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 8, a video call device based on a browser according to an embodiment of the present invention is shown, where the device may be located on a video networking server in a video call system, the video call system may further include a mobile terminal and a PC terminal, and the PC terminal runs on the browser; the device may specifically include the following modules:

a video receiving module 801, configured to receive, in a video call between the PC and the mobile terminal, first video data sent by the browser, and receive third video data sent by the mobile terminal;

a video transcoding module 802, configured to convert the first video data into second video data conforming to a first preset video coding protocol, and convert the third video data into fourth video data conforming to a second preset video coding protocol;

a video sending module 803, configured to send the second video data to the mobile terminal, and send the fourth video data to the browser;

Optionally, the video transcoding module 802 may specifically include the following units:

the second encoding unit may be configured to encode the third original video data according to the second preset video encoding protocol to obtain the fourth video data.

Optionally, the video receiving module 801 may be specifically configured to receive the first video data sent by the browser based on an RTP protocol or an RTCP protocol;

the video sending module 803 may be specifically configured to send the fourth video data to the browser based on the RTP protocol or the RTCP protocol.

Optionally, the video call system further includes a first server and a second server. Wherein:

the video receiving module 801 may be specifically configured to receive first video data sent by the first server and receive third video data sent by the second server, where the first video data is sent to the first server by the browser, and the third video data is sent to the second server by the mobile terminal.

The video transcoding module 802 may further include the following units:

the first protocol conversion unit is used for encoding the first video data into first transcoding video data conforming to a second preset video encoding protocol and adding a video networking protocol packet header to the first transcoding video data to obtain second video data;

the second protocol conversion unit is used for encoding the third video data into second transcoded video data conforming to a second preset video encoding protocol and adding an internet protocol packet header to the second transcoded video data to obtain fourth video data;

the video receiving module 801 may be specifically configured to send the second video data to the second server, and send the fourth video data to the first server. The second server is used for sending the second video data to the mobile terminal, and the first server is used for sending the fourth video data to the browser.

Optionally, in the apparatus, the first predetermined video coding protocol may be a VP8 protocol, and the second predetermined video coding protocol may be an h.264 video protocol.

For the embodiment of the video call device based on the browser, since it is basically similar to the embodiment of the video call method based on the browser, the description is relatively simple, and for the relevant points, refer to the partial description of the embodiment of the method of the video call based on the browser.

An embodiment of the present invention further provides an electronic device, including: one or more processors; and one or more machine readable media having instructions stored thereon, which when executed by the one or more processors, cause the apparatus to perform one or more browser-based video call methods as described in embodiments of the invention.

Embodiments of the present invention further provide a computer-readable storage medium, which stores a computer program to enable a processor to execute the video call method based on a browser according to the embodiments of the present invention.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The video call method, the video call device, the electronic device and the storage medium based on the browser provided by the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation of the invention, and the description of the above embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A video call method based on a browser is characterized in that: the method is applied to a video networking server in a video call system, the video call system also comprises a mobile terminal and a PC end, and a browser runs on the PC end; the method comprises the following steps:

2. The method of claim 1, wherein converting the first video data into second video data compliant with a first predetermined video coding protocol comprises:

decoding the first video data to obtain first original video data;

decoding the third video data to obtain third original video data;

3. The method of claim 1, wherein receiving the first video data sent by the browser comprises:

sending the fourth video data to the browser, including:

4. The method of claim 1, further comprising a first server and a second server in the video call system; receiving first video data sent by the browser, and converting the first video data into second video data conforming to a first preset video coding protocol; and receiving third video data sent by the mobile terminal, and converting the third video data into fourth video data conforming to a second preset video coding protocol, including:

5. The method of claim 1, wherein the first predetermined video coding protocol is a VP8 protocol, and wherein the second predetermined video coding protocol is a h.264 video protocol.

6. A video call device based on browser is characterized in that: the device is positioned in a video networking server in a video call system, the video call system also comprises a mobile terminal and a PC (personal computer) end, and a browser runs on the PC end; the device comprises:

7. The apparatus of claim 6, wherein the video transcoding module comprises:

8. The apparatus according to claim 6, wherein the video receiving module is specifically configured to receive the first video data sent by the browser based on an RTP protocol or an RTCP protocol;

9. An electronic device, comprising:

one or more processors; and

one or more machine-readable media having instructions stored thereon that, when executed by the one or more processors, cause the device to perform one or more of the browser-based video call methods of claims 1-5.

10. A computer-readable storage medium storing a computer program for causing a processor to execute the browser-based video call method according to any one of claims 1 to 5.