CN110213235A - The method and system of iOS device and view networked terminals video calling - Google Patents

Abstract

The embodiment of the invention provides the method and systems of a kind of iOS device and view networked terminals video calling, the implementation method of a kind of iOS device of the present invention and view networked terminals video calling, applied to view networked system, which comprises iOS device and depending on networked terminals establish connection；IOS device acquires the first video source data and the first audio source data；IOS device carries out coding to the first video source data and forms the first video frame, carries out coding to the first audio source data and forms the first audio frame；First video frame and the first audio frame are packaged as the first audio, video data packet by iOS device；First audio, video data packet is sent to streaming media server by iOS device, so that the first audio, video data packet is transmitted to the view networked terminals by streaming media server.The present invention solves in the prior art, and iOS device cannot be with view networked terminals video calling, the bad problem of user experience.

Description

The method and system of iOS device and view networked terminals video calling

Technical field

The present invention relates to view networking technology fields, more particularly to a kind of iOS device and regard networked terminals video calling Method and system.

Background technique

With the fast development of the network technology, such as video conference, video teaching are in life, work, study of user etc. Aspect is widely available.

It is the more advanced form of internet depending on networking, is a real-time network, can be realized current internet cannot achieve The whole network HD video real-time Transmission, push numerous Internet applications to HD video, high definition is face-to-face.

Due to regarding the limitation of networking protocol, in the prior art, iOS device cannot be with view networked terminals video calling, user It experiences bad.

Summary of the invention

In view of the above problems, it proposes the embodiment of the present invention and overcomes the above problem or at least partly in order to provide one kind The method and system of a kind of iOS device and view networked terminals video calling that solve the above problems.

To solve the above-mentioned problems, the embodiment of the invention discloses one kind to solve the above-mentioned problems, the embodiment of the present invention The implementation method of a kind of iOS device and view networked terminals video calling is disclosed, the method is applied to view networked system, described It include iOS device, streaming media server and view networked terminals depending on networked system, the iOS device is logged in by first network to flow Media server, the view networked terminals log in streaming media server by view networking, and the first network is different from the view Networking, which comprises

The iOS device sends video calling request or the view networked terminals to the iOS to the view networked terminals Equipment sends video calling request, so as to the iOS device and described establish connection depending on networked terminals；

The iOS device acquires the first video source data and the first audio source data；

The iOS device carries out coding according to the first coding mode to first video source data and forms the first video Frame carries out coding according to the second coding mode to the first audio source data and forms the first audio frame, first coding staff Formula is videotoolbox coding, and second coding mode is audiotoolbox coding；

First video frame and first audio frame are packaged as the first audio, video data packet by the iOS device, hair The streaming media server is given, is joined so that the first audio, video data packet is transmitted to the view by the streaming media server Network termination.

Further, the method also includes:

The view networked terminals receive the first audio, video data packet, obtain the in the first audio, video data packet One video frame and the first audio frame decode first video frame and first audio frame and broadcast described depending on networked terminals It puts.

Further, the method also includes:

The view networked terminals acquire the second video source data and the second audio source data；

It is described that second video source data is carried out by coding the second video frame of formation using FFmpeg depending on networked terminals, it adopts The second audio source data is subjected to coding with FFmpeg and forms the second audio frame；

It is described that second video frame and second audio frame are packaged as the second audio, video data packet depending on networked terminals, Be sent to the streaming media server so that the streaming media server the second audio, video data packet is transmitted to it is described IOS device.

Further, the method also includes:

The iOS device receives the second audio, video data packet, obtains second in the second audio, video data packet Video frame and the second audio frame decode second video frame and second audio frame and play in the iOS device.

Further, first video source data is yuv format, and the first audio source data is PCM format, described First video frame is that H.264 format, first audio frame format are AAC format.

Further, second video source data is yuv format, and the second audio source data is PCM format, described Second video frame is that H.264 format, second audio frame format are AAC format.

The embodiment of the invention also discloses a kind of video callings to realize system, comprising:

IOS device, streaming media server and view networked terminals；

The iOS device sends video to the view networked terminals for logging in streaming media server by first network Call request receives the video calling request that the view networked terminals are sent, so that the iOS device and view networking are eventually Connection is established at end, and the first network is different from view networking；

The iOS device is also used to acquire the first video source data and the first audio source data, to first video source Data carry out coding according to the first coding mode and form the first video frame, to the first audio source data according to the second coding staff Formula carries out coding and forms the first audio frame, and first coding mode is VideoToolBox coding, second coding mode For AudioToolBox coding, first video frame and first audio frame are packaged as the first audio, video data packet, sent out Give the streaming media server；

The streaming media server, for the first audio, video data packet to be transmitted to the view networked terminals；

The view networked terminals receive first audio, video data for logging in streaming media server by view networking Packet.

Further, the view networked terminals are also used to obtain the first video frame in the first audio, video data packet With the first audio frame, decodes first video frame and first audio frame and played in the view networked terminals.

Further, the view networked terminals are also used to acquire the second video source data and the second audio source data, use Second video source data is carried out coding and forms the second video frame by FFmpeg, using FFmpeg by the second audio-source number The second audio frame is formed according to coding is carried out, second video frame and second audio frame are packaged as the second audio, video data Packet, is sent to the streaming media server；

The streaming media server is also used to the second audio, video data packet being transmitted to the iOS device.

Further, the iOS device is also used to receive the second audio, video data packet, obtains the second sound view The second video frame and the second audio frame in frequency data packet decode second video frame and second audio frame and described IOS device plays.

The embodiment of the present invention includes following advantages:

IOS device and view networked terminals are connected by streaming media server in the present invention, iOS device is joined with view Network termination carries out video calling.It solves in the prior art, iOS device cannot be with view networked terminals video calling, user experience Bad problem so that user can be convenient by iOS device with view networked terminals carry out video calling, improve user's body It tests.

Detailed description of the invention

Fig. 1 is a kind of networking schematic diagram of view networking of the invention；

Fig. 2 is a kind of hardware structural diagram of node server of the invention；

Fig. 3 is a kind of hardware structural diagram of access switch of the invention；

Fig. 4 is the hardware structural diagram that a kind of Ethernet association of the invention turns gateway；

The step of Fig. 5 is a kind of iOS device of the present invention and regards the implementation method embodiment one of networked terminals video calling stream Cheng Tu；

The step of Fig. 6 is a kind of iOS device of the present invention and regards the implementation method embodiment two of networked terminals video calling stream Cheng Tu；

Fig. 7 is the structural schematic diagram that a kind of video calling of the present invention realizes system embodiment one.

Specific embodiment

In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, with reference to the accompanying drawing and specific real Applying mode, the present invention is described in further detail.

It is the important milestone of network Development depending on networking, is a real-time network, can be realized HD video real-time Transmission, Push numerous Internet applications to HD video, high definition is face-to-face.

Real-time high-definition video switching technology is used depending on networking, it can be such as high in a network platform by required service Clear video conference, Intellectualized monitoring analysis, emergency command, digital broadcast television, delay TV, the Web-based instruction, shows video monitoring Field live streaming, VOD program request, TV Mail, individual character records (PVR), Intranet (manages) channel by oneself, intelligent video Broadcast Control, information publication All be incorporated into a system platform etc. services such as tens of kinds of videos, voice, picture, text, communication, data, by TV or Computer realizes that high-definition quality video plays.

Embodiment in order to enable those skilled in the art to better understand the present invention is introduced to depending on networking below:

Depending on networking, applied portion of techniques is as described below:

Network technology (Network Technology)

Traditional ethernet (Ethernet) is improved depending on the network technology innovation networked, with potential huge on network First video flow.It is exchanged different from simple network packet packet switch (Packet Switching) or lattice network (Circuit Switching) meets Streaming demand using Packet Switching depending on networking technology.Depending on skill of networking Art has flexible, the simple and low price of packet switch, is provided simultaneously with the quality and safety assurance of circuit switching, realizes the whole network friendship Change the seamless connection of formula virtual circuit and data format.

Switching technology (Switching Technology)

Two advantages of asynchronous and packet switch that Ethernet is used depending on networking eliminate Ethernet under the premise of complete compatible and lack It falls into, has the end-to-end seamless connection of the whole network, direct user terminal, directly carrying IP data packet.User data is in network-wide basis It is not required to any format conversion.It is the more advanced form of Ethernet depending on networking, is a real-time exchange platform, can be realized at present mutually The whole network large-scale high-definition realtime video transmission that networking cannot achieve pushes numerous network video applications to high Qinghua, unitizes.

Server technology (Server Technology)

It is different from traditional server, its Streaming Media depending on the server technology in networking and unified video platform Transmission be built upon it is connection-oriented on the basis of, data-handling capacity is unrelated with flow, communication time, single network layer energy Enough transmitted comprising signaling and data.For voice and video business, handled depending on networking and unified video platform Streaming Media Complexity many simpler than data processing, efficiency substantially increase hundred times or more than traditional server.

Reservoir technology (Storage Technology)

The ultrahigh speed reservoir technology of unified video platform in order to adapt to the media content of vast capacity and super-flow and Using state-of-the-art real time operating system, the programme information in server instruction is mapped to specific hard drive space, media Content is no longer pass through server, and moment is directly delivered to user terminal, and user waits typical time less than 0.2 second.It optimizes Sector distribution greatly reduces the mechanical movement of hard disc magnetic head tracking, and resource consumption only accounts for the 20% of the internet ad eundem IP, but The concurrent flow greater than 3 times of traditional disk array is generated, overall efficiency promotes 10 times or more.

Network security technology (Network Security Technology)

Depending on the structural design networked by servicing independent licence system, equipment and the modes such as user data is completely isolated every time The network security problem that puzzlement internet has thoroughly been eradicated from structure, does not need antivirus applet, firewall generally, has prevented black The attack of visitor and virus, structural carefree secure network is provided for user.

It services innovative technology (Service lnnovation Technology)

Business and transmission are fused together by unified video platform, whether single user, private user or a net The sum total of network is all only primary automatic connection.User terminal, set-top box or PC are attached directly to unified video platform, obtain rich The multimedia video service of rich colorful various forms.Unified video platform is traditional to substitute with table schema using " menu type " Complicated applications programming, considerably less code, which can be used, can be realized complicated application, realize the new business innovation of " endless ".

Networking depending on networking is as described below:

It is a kind of central controlled network structure depending on networking, which can be Tree Network, Star network, ring network etc. class Type, but centralized control node is needed to control whole network in network on this basis.

As shown in Figure 1, being divided into access net and Metropolitan Area Network (MAN) two parts depending on networking.

The equipment of access mesh portions can be mainly divided into 3 classes: node server, access switch, terminal (including various machines Top box, encoding board, memory etc.).Node server is connected with access switch, and access switch can be with multiple terminal phases Even, and it can connect Ethernet.

Wherein, node server is the node that centralized control functions are played in access net, can control access switch and terminal. Node server can directly be connected with access switch, can also directly be connected with terminal.

Similar, the equipment of metropolitan area mesh portions can also be divided into 3 classes: metropolitan area server, node switch, node serve Device.Metropolitan area server is connected with node switch, and node switch can be connected with multiple node servers.

Wherein, node server is the node server for accessing mesh portions, i.e. node server had both belonged to access wet end Point, and belong to metropolitan area mesh portions.

Metropolitan area server is the node that centralized control functions are played in Metropolitan Area Network (MAN), can control node switch and node serve Device.Metropolitan area server can be directly connected to node switch, can also be directly connected to node server.

It can be seen that be entirely a kind of central controlled network structure of layering depending on networking network, and node server and metropolitan area The network controlled under server can be the various structures such as tree-shaped, star-like, cyclic annular.

Visually claim, access mesh portions can form unified video platform (part in virtual coil), and multiple unified videos are flat Platform can form view networking；Each unified video platform can be interconnected by metropolitan area and wide area depending on networking.

Classify depending on networked terminals

1.1 embodiment of the present invention can be mainly divided into 3 classes: server depending on the equipment in networking, interchanger (including ether Net association turns gateway), terminal (including various set-top boxes, encoding board, memory etc.).Metropolitan Area Network (MAN) can be divided on the whole depending on networking (or national net, World Wide Web etc.) and access net.

1.2 equipment for wherein accessing mesh portions can be mainly divided into 3 classes: node server, access switch (including ether Net association turns gateway), terminal (including various set-top boxes, encoding board, memory etc.).

The specific hardware structure of each access network equipment are as follows:

Node server:

As shown in Fig. 2, mainly including Network Interface Module 201, switching engine module 202, CPU module 203, disk array Module 204；

Wherein, Network Interface Module 201, the Bao Jun that CPU module 203, disk array module 204 are come in enter switching engine Module 202；Switching engine module 202 look into the operation of address table 205 to the packet come in, to obtain the navigation information of packet； And the packet is stored according to the navigation information of packet the queue of corresponding pack buffer 206；If the queue of pack buffer 206 is close It is full, then it abandons；All pack buffer queues of 202 poll of switching engine module, are forwarded: 1) end if meeting the following conditions It is less than that mouth sends caching；2) the queue package counting facility is greater than zero.Disk array module 204 mainly realizes the control to hard disk, packet Include the operation such as initialization, read-write to hard disk；CPU module 203 be mainly responsible for access switch, terminal (not shown) it Between protocol processes, to address table 205 (including descending protocol packet address table, uplink protocol package address table, data packet addressed table) Configuration, and, the configuration to disk array module 204.

Access switch:

As shown in figure 3, mainly including Network Interface Module (downstream network interface module 301, uplink network interface module 302), switching engine module 303 and CPU module 304；

Wherein, the packet (upstream data) that downstream network interface module 301 is come in enters packet detection module 305；Packet detection mould Whether mesh way address (DA), source address (SA), type of data packet and the packet length of the detection packet of block 305 meet the requirements, if met, It then distributes corresponding flow identifier (stream-id), and enters switching engine module 303, otherwise abandon；Uplink network interface mould The packet (downlink data) that block 302 is come in enters switching engine module 303；The data packet that CPU module 304 is come in enters switching engine Module 303；Switching engine module 303 look into the operation of address table 306 to the packet come in, to obtain the navigation information of packet； If the packet into switching engine module 303 is that downstream network interface is gone toward uplink network interface, in conjunction with flow identifier (stream-id) packet is stored in the queue of corresponding pack buffer 307；If the queue of the pack buffer 307 is close full, It abandons；If the packet into switching engine module 303 is not that downstream network interface is gone toward uplink network interface, according to packet Navigation information is stored in the data packet queue of corresponding pack buffer 307；If the queue of the pack buffer 307 is close full, Then abandon.

All pack buffer queues of 303 poll of switching engine module, may include two kinds of situations:

If the queue is that downstream network interface is gone toward uplink network interface, meets the following conditions and be forwarded: 1) It is less than that the port sends caching；2) the queue package counting facility is greater than zero；3) token that rate control module generates is obtained；

If the queue is not that downstream network interface is gone toward uplink network interface, meets the following conditions and is forwarded: 1) it is less than to send caching for the port；2) the queue package counting facility is greater than zero.

Rate control module 308 is configured by CPU module 304, to all downlink networks in programmable interval Interface generates token toward the pack buffer queue that uplink network interface is gone, to control the code rate of forwarded upstream.

CPU module 304 is mainly responsible for the protocol processes between node server, the configuration to address table 306, and, Configuration to rate control module 308.

Ethernet association turns gateway:

As shown in figure 4, mainly including Network Interface Module (downstream network interface module 401, uplink network interface module 402), switching engine module 403, CPU module 404, packet detection module 405, rate control module 408, address table 406, Bao Huan Storage 407 and MAC adding module 409, MAC removing module 410.

Wherein, the data packet that downstream network interface module 401 is come in enters packet detection module 405；Packet detection module 405 is examined Ethernet mac DA, ethernet mac SA, Ethernet length or frame type, the view networking mesh way address of measured data packet DA, whether meet the requirements depending on networking source address SA, depending on networking data Packet type and packet length, corresponding stream is distributed if meeting Identifier (stream-id)；Then, MAC DA, MAC SA, length or frame type are subtracted by MAC removing module 410 (2byte), and enter corresponding receive and cache, otherwise abandon；

Downstream network interface module 401 detects the transmission caching of the port, according to the view of packet networking mesh if there is Bao Ze Address D A knows the ethernet mac DA of corresponding terminal, adds the ethernet mac DA of terminal, Ethernet assists the MAC for turning gateway SA, Ethernet length or frame type, and send.

The function that Ethernet association turns other modules in gateway is similar with access switch.

Terminal:

It mainly include Network Interface Module, Service Processing Module and CPU module；For example, set-top box mainly connects including network Mouth mold block, video/audio encoding and decoding engine modules, CPU module；Encoding board mainly includes Network Interface Module, video encoding engine Module, CPU module；Memory mainly includes Network Interface Module, CPU module and disk array module.

The equipment of 1.3 metropolitan area mesh portions can be mainly divided into 2 classes: node server, node switch, metropolitan area server. Wherein, node switch mainly includes Network Interface Module, switching engine module and CPU module；Metropolitan area server mainly includes Network Interface Module, switching engine module and CPU module are constituted.

2, networking data package definition is regarded

2.1 access network data package definitions

Access net data packet mainly include following sections: destination address (DA), source address (SA), reserve bytes, payload(PDU)、CRC。

As shown in the table, the data packet for accessing net mainly includes following sections:

DA

SA

Reserved

Payload

CRC

Wherein:

Destination address (DA) is made of 8 bytes (byte), and first character section indicates type (such as the various associations of data packet Discuss packet, multicast packet, unicast packet etc.), be up to 256 kinds of possibility, the second byte to the 6th byte is metropolitan area net address, Seven, the 8th bytes are access net address；

Source address (SA) is also to be made of 8 bytes (byte), is defined identical as destination address (DA)；

Reserve bytes are made of 2 bytes；

The part payload has different length according to the type of different datagrams, is if it is various protocol packages 64 bytes are 32+1024=1056 bytes if it is single group unicast packets words, are not restricted to above 2 kinds certainly；

CRC is made of 4 bytes, and calculation method follows the Ethernet CRC algorithm of standard.

2.2 Metropolitan Area Network (MAN) packet definitions

The topology of Metropolitan Area Network (MAN) is pattern, may there is 2 kinds, connection even of more than two kinds, i.e. node switching between two equipment It can all can exceed that 2 kinds between machine and node server, node switch and node switch, node switch and node server Connection.But the metropolitan area net address of metropolitan area network equipment is uniquely, to close to accurately describe the connection between metropolitan area network equipment System, introduces parameter in embodiments of the present invention: label, uniquely to describe a metropolitan area network equipment.

(Multi-Protocol Label Switch, multiprotocol label are handed over by the definition of label and MPLS in this specification Change) label definition it is similar, it is assumed that between equipment A and equipment B there are two connection, then data packet from equipment A to equipment B just There are 2 labels, data packet also there are 2 labels from equipment B to equipment A.Label is divided into label, outgoing label, it is assumed that data packet enters The label (entering label) of equipment A is 0x0000, and the label (outgoing label) when this data packet leaves equipment A may reform into 0x0001.The networking process of Metropolitan Area Network (MAN) is to enter network process under centralized control, also means that address distribution, the label of Metropolitan Area Network (MAN) Distribution be all to be dominated by metropolitan area server, node switch, node server be all passively execute, this point with The label distribution of MPLS is different, and the distribution of the label of MPLS is the result that interchanger, server are negotiated mutually.

As shown in the table, the data packet of Metropolitan Area Network (MAN) mainly includes following sections:

DA

SA

Reserved

Label

Payload

CRC

That is destination address (DA), source address (SA), reserve bytes (Reserved), label, payload (PDU), CRC.Its In, the format of label, which can refer to, such as gives a definition: label is 32bit, wherein high 16bit retains, only with low 16bit, its position Set is between the reserve bytes and payload of data packet.

With reference to Fig. 5, the implementation method embodiment one of a kind of iOS device of the present invention and view networked terminals video calling is shown Step flow chart, the method is applied to view networked system, and described depending on networked system includes iOS device, streaming media server With view networked terminals, the iOS device logs in streaming media server by first network, and the view networked terminals pass through view networking Streaming media server is logged in, the first network is networked different from the view；

In the present embodiment, iOS device can be iOS mobile phone, iOS tablet computer, the equipment such as iOS TV.

Specifically, first network is different from view networking, first network can be internet or mobile communications network, the present invention It is not specifically limited in this embodiment.As iOS mobile phone can pass through mobile communications network or wifi connection streaming media server, iOS plate electricity Brain or iOS TV can connect streaming media server by internet.

The method can specifically include following steps:

S501, iOS device send video calling request or the view networked terminals to the iOS to the view networked terminals Equipment sends video calling request, so as to the iOS device and described establish connection depending on networked terminals；

In the present embodiment, specifically, iOS device or the video calling request sent depending on networked terminals can first be sent to stream matchmaker Body server is transmitted to target side by streaming media server.

Streaming media server is the bridge and tie of view networking and internet audio video transmission business, realizes view networking Business is merged with the seamless of interconnection services, audio and video resources secure accessings all kinds of in internet can be regarded and be networked, and can join view The audio/video flow that video conference, monitoring image, DTV etc. are different in net is converted and with the Internet protocol side of working standard Formula output is depending on the core equipment to interconnect with internet net cell phone client of networking.

Optionally, streaming media server can also support mobile communications network, be connected by mobile communications network and terminal device It connects.

In the present embodiment, iOS device is made by streaming media server and establishes connection depending on networked terminals, so that IOS device can also be with view networked terminals video calling.

Optionally, iOS device sends video calling request to view networked terminals, by three-way handshake agreement and regards networking eventually Connection is established at end.

Three-way handshake agreement refer to send data preparation stage, iOS device and depending on needed between networked terminals into Row interacts three times: shake hands for the first time: iOS device sends solicited message to view networked terminals, waits to be confirmed；Second handshake: view Networked terminals receive solicited message, also send second information to iOS device；Third time is shaken hands: iOS device receives view networking Second information of terminal sends confirmation message to server, completes three-way handshake.After connection is established, client and server is just It can start to carry out data transmission.

Similarly, sending when video calling is requested depending on networked terminals to iOS device is also same process.

S502, iOS device acquire the first video source data and the first audio source data；

When video calling, the data of iOS device side are acquired by iOS device, and the data of iOS device acquisition are divided into the first video Source data and the first audio source data, specifically, the first video source data is yuv format, the first audio source data is PCM lattice Formula.

YUV: video is to be formed by data connection one by one, and one-frame video data is exactly a picture in fact.YUV It is a kind of picture saving format, it is similar with rgb format.

PCM: when being exactly recorded voice, the voice data format of the most original of preservation.

S503, iOS device carries out coding according to the first coding mode to the first video source data and forms the first video frame, right First audio source data carries out coding according to the second coding mode and forms the first audio frame；

Optionally, the first coding mode is videotoolbox coding, and the second coding mode is audiotoolbox coding；

Videotoolbox coding and audiotoolbox coding are hard encoding and decoding, relative to soft coding saving CPU and interior It deposits.

VideoToolBox is the application programming interface (Application for hard decoder encoded video Programming Interface, API), AudioToolbox is the application programming interface for encoding and decoding audio (Application Programming Interface, API).

Optionally, first video frame is that H.264 format, first audio frame format are AAC format.

It is the first video frame of H.264 format by the first video frequency source coding of yuv format, by PCM format in the present embodiment The first audio-source data encoding be AAC format the first audio frame format.

H.264:H.264 target application covers current most Video service, and such as wired television remote monitoring is handed over Mutual media, DTV, video conference, video on demand, streaming media service etc..H.264 the network to solve in different application passes Defeated difference.Define two layers: video coding layer is responsible for efficient video frequency content expressing, and network abstraction layer is responsible for being wanted with network The appropriate mode asked is packaged and is transmitted to data.

AAC: being a kind of file compression formats for aiming at voice data design.Different from MP3, it uses completely new algorithm It is encoded, more efficiently, has that higher " cost performance is copied.Using AAC format, it is unobvious that perceived sounds quality can be made one It is more compact under the premise of reduction.Apple ipod, Nokia's mobile phone support the audio file of AAC format.

First video frame and the first audio frame are packaged as the first audio, video data packet by S504, iOS device；

Specifically, by audio-video video data according to certain format package, the present invention is not specifically limited in this embodiment.

S505, the first audio, video data packet is sent to the streaming media server by the iOS device, so that described The first audio, video data packet is transmitted to the view networked terminals by streaming media server.

Specifically, the first audio, video data is transmitted to and realizes an iOS depending on networked terminals and set by streaming media server It is standby view networked terminals between audio/video information transmission, enable iOS device and view networked terminals free communication.

Optionally, the method can also include:

S506, the view networked terminals receive first audio, video data, obtain in the first audio, video data packet The first video frame and the first audio frame, decode first video frame and first audio frame and broadcast depending on networked terminals It puts.

Specifically, the first video frame of H.264 format is decoded as yuv format depending on networked terminals, by the first of AAC format Audio frame is decoded as PCM format.

Optionally, iOS device is encoded by the first tandem queue, receives view networked terminals by the second tandem queue Audio data is decoded into PCM data, is then played back by loudspeaker, by video data by the audio, video data sended over It is decoded into yuv data, and is rendered by OpenGL.

Open graphic library (Open Graphics Library, OpenGL) be for render 2D, 3D vector graphics across language Speech, cross-platform application programming interface (API).

In the present embodiment, by streaming media server connect iOS device and view networked terminals, allow iOS device with Video calling is carried out depending on networked terminals, is solved in the prior art, iOS device cannot be with view networked terminals video calling, user Experience bad problem so that user can be convenient by iOS device with view networked terminals carry out video calling, improve use Family experience.

With reference to Fig. 6, the implementation method embodiment two of a kind of iOS device of the present invention and view networked terminals video calling is shown Step flow chart, on the basis of embodiment illustrated in fig. 6 is embodiment shown in Fig. 5, embodiment illustrated in fig. 6 is further illustrated Depending on the audio-video collection and process flow of networked terminals equipment end.

Specifically, the described method includes:

S601 acquires the second video source data and the second audio source data depending on networked terminals；

In the present embodiment, for the data depending on networked terminals side by acquiring depending on networked terminals, the data of iOS device acquisition are divided into the Two video source datas and the second audio source data, specifically, the second video source data is yuv format, the second audio source data is PCM format.

Second video source data is carried out coding using FFmpeg depending on networked terminals and forms the second video frame by S602, The second audio source data is carried out by coding using FFmpeg and forms the second audio frame；

It is the second video frame of H.264 format by the second video frequency source coding of yuv format, by PCM format in the present embodiment The second audio-source data encoding be AAC format the second audio frame format.

FFmpeg be it is a set of can be used to record, converted digital audio, video, and can be translated into stream open source calculate Machine program.

S603, it is described that second video frame and second audio frame are packaged as the second audio-video number depending on networked terminals According to packet；

Specifically, by audio-video video data according to certain format package, the present invention is not specifically limited in this embodiment.

S604, it is described that the second audio, video data packet is sent to the streaming media server depending on networked terminals, so that The second audio, video data packet is transmitted to the iOS device by the streaming media server；

S605, iOS device receive the second audio, video data packet, obtain second in the second audio, video data packet Video frame and the second audio frame decode second video frame and second audio frame and play in the iOS device.

Specifically, the second video frame of H.264 format is decoded as yuv format depending on networked terminals, by the second of AAC format Audio frame is decoded as PCM format.

In the present embodiment, iOS device and view networked terminals are connected by streaming media server, in iOS and view networked terminals On using different coding modes, realize that iOS device can carry out video calling with view networked terminals.Solves the prior art In, iOS device cannot with view networked terminals video calling, the bad problem of user experience so that user can be with by iOS device Video calling easily is carried out with view networked terminals, improves user experience.

It should be noted that for simple description, therefore, it is stated as a series of action groups for embodiment of the method It closes, but those skilled in the art should understand that, embodiment of that present invention are not limited by the describe sequence of actions, because according to According to the embodiment of the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art also should Know, the embodiments described in the specification are all preferred embodiments, and the related movement not necessarily present invention is implemented Necessary to example.

With reference to Fig. 7, the structural schematic diagram that a kind of video calling of the present invention realizes system embodiment one: the system is shown Include:

IOS device 71, streaming media server 72 and view networked terminals 73,

The iOS device 71 sends to the view networked terminals and regards for logging in streaming media server by first network Frequency call request receives the video calling request that the view networked terminals are sent, so that the iOS device and view networking Terminal establishes connection, and the first network is different from view networking；

The iOS device 71 is also used to acquire the first video source data and the first audio source data, to first video Source data carries out coding according to the first coding mode and forms the first video frame, to the first audio source data according to the second coding Mode carries out coding and forms the first audio frame, and first coding mode is VideoToolBox coding, second coding staff Formula is AudioToolBox coding, and first video frame and first audio frame are packaged as the first audio, video data packet, It is sent to the streaming media server,

The streaming media server 72, for the first audio, video data packet to be transmitted to the view networked terminals；

The view networked terminals 73 receive the first audio-video number for logging in streaming media server by view networking According to packet.

Optionally, the first video frame for being also used to obtain in the first audio, video data packet depending on networked terminals 73 and One audio frame decodes first video frame and first audio frame and plays in the view networked terminals.

Optionally, it depending on networked terminals 73, is also used to acquire the second video source data and the second audio source data, uses Second video source data is carried out coding and forms the second video frame by FFmpeg, using FFmpeg by the second audio-source number The second audio frame is formed according to coding is carried out；

The view networked terminals 73 are also used to for second video frame and second audio frame being packaged as the second sound view Frequency data packet is sent to the streaming media server；

The streaming media server 72 is also used to the second audio, video data packet being transmitted to the iOS device.

Optionally, the iOS device 71 is also used to receive the second audio, video data packet, obtains the second sound view The second video frame and the second audio frame in frequency data packet decode second video frame and second audio frame and described IOS device plays.

Above-mentioned video calling realizes that system can be used for executing iOS device shown in Fig. 5-6 and regard networked terminals video calling Implementation method.

For video call system embodiment, due to the realization of itself and iOS device and view networked terminals video calling Embodiment of the method is substantially similar, so being described relatively simple, related place is referring to iOS device and view networked terminals video calling Implementation method embodiment part explanation.

All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.

It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can provide as method, apparatus or calculate Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code The form of the computer program product of implementation.

The embodiment of the present invention be referring to according to the method for the embodiment of the present invention, terminal device (system) and computer program The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these Computer program instructions are set to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals Standby processor is to generate a machine, so that being held by the processor of computer or other programmable data processing terminal devices Capable instruction generates for realizing in one or more flows of the flowchart and/or one or more blocks of the block diagram The device of specified function.

These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing terminal devices In computer-readable memory operate in a specific manner, so that instruction stored in the computer readable memory generates packet The manufacture of command device is included, which realizes in one side of one or more flows of the flowchart and/or block diagram The function of being specified in frame or multiple boxes.

These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that Series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented processing, thus The instruction executed on computer or other programmable terminal equipments is provided for realizing in one or more flows of the flowchart And/or in one or more blocks of the block diagram specify function the step of.

Although the preferred embodiment of the embodiment of the present invention has been described, once a person skilled in the art knows bases This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.

Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including the element.

It is carried out above to iOS device provided by the present invention and depending on the method and system of networked terminals video calling It is discussed in detail, used herein a specific example illustrates the principle and implementation of the invention, above embodiments Illustrate to be merely used to help understand method and its core concept of the invention；At the same time, for those skilled in the art, according to According to thought of the invention, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification It should not be construed as limiting the invention.

Claims (10)

1. the implementation method of a kind of iOS device and view networked terminals video calling, which is characterized in that the method is applied to view connection Net system, it is described depending on networked system include iOS device, streaming media server and view networked terminals, the iOS device pass through first Network logs in the streaming media server, and the view networked terminals log in streaming media server, first net by view networking Network is networked different from the view, which comprises

The iOS device sends video calling request or the view networked terminals to the iOS device to the view networked terminals Video calling request is sent, so as to the iOS device and described establish connection depending on networked terminals；

The iOS device acquires the first video source data and the first audio source data；

The iOS device carries out coding according to the first coding mode to first video source data and forms the first video frame, right The first audio source data carries out coding according to the second coding mode and forms the first audio frame, and first coding mode is Videotoolbox coding, second coding mode are audiotoolbox coding；

First video frame and first audio frame are packaged as the first audio, video data packet by the iOS device；

The first audio, video data packet is sent to the streaming media server by the iOS device, so that the Streaming Media takes The first audio, video data packet is transmitted to the view networked terminals by business device.

2. implementation method according to claim 1, which is characterized in that the method also includes:

The view networked terminals receive the first audio, video data packet, obtain the first view in the first audio, video data packet Frequency frame and the first audio frame decode first video frame and first audio frame and play in the view networked terminals.

3. implementation method according to claim 1 or 2, which is characterized in that the method also includes:

The view networked terminals acquire the second video source data and the second audio source data；

It is described that second video source data is carried out by coding the second video frame of formation using FFmpeg depending on networked terminals, it uses The second audio source data is carried out coding and forms the second audio frame by FFmpeg；

It is described that second video frame and second audio frame are packaged as the second audio, video data packet depending on networked terminals；

It is described that the second audio, video data packet is sent to the streaming media server depending on networked terminals, so that the Streaming Media The second audio, video data packet is transmitted to the iOS device by server.

4. according to the method described in claim 3, it is characterized in that, the method also includes:

The iOS device receives the second audio, video data packet, obtains the second video in the second audio, video data packet Frame and the second audio frame decode second video frame and second audio frame and play in the iOS device.

5. method according to claim 1 or 2, which is characterized in that first video source data is yuv format, described First audio source data is PCM format, and first video frame is that H.264 format, first audio frame format are AAC lattice Formula.

6. according to the method described in claim 3, it is characterized in that, second video source data be yuv format, described second Audio source data is PCM format, and second video frame is that H.264 format, second audio frame format are AAC format.

7. a kind of video calling realizes system characterized by comprising

IOS device, streaming media server and view networked terminals；

The iOS device sends video calling to the view networked terminals for logging in streaming media server by first network The video calling request that the view networked terminals are sent is requested or receives, so as to the iOS device and described build depending on networked terminals Vertical connection, the first network are different from view networking；

The iOS device is also used to acquire the first video source data and the first audio source data, to first video source data According to the first coding mode carry out coding formed the first video frame, to the first audio source data according to the second coding mode into Row coding forms the first audio frame, and first coding mode is VideoToolBox coding, and second coding mode is AudioToolBox coding, is packaged as the first audio, video data packet for first video frame and first audio frame, sends To the streaming media server；

The streaming media server, for the first audio, video data packet to be transmitted to the view networked terminals；

The view networked terminals receive the first audio, video data packet for logging in streaming media server by view networking.

8. system according to claim 7, which is characterized in that

The view networked terminals, the first video frame for being also used to obtain in the first audio, video data packet and the first audio frame, It decodes first video frame and first audio frame and is played in the view networked terminals.

9. system according to claim 7 or 8, which is characterized in that

The view networked terminals are also used to acquire the second video source data and the second audio source data, will be described using FFmpeg Second video source data carries out coding and forms the second video frame, and the second audio source data is carried out coding shape using FFmpeg At the second audio frame, second video frame and second audio frame are packaged as the second audio, video data packet, are sent to institute State streaming media server；

The streaming media server is also used to the second audio, video data packet being transmitted to the iOS device.

10. system according to claim 9, which is characterized in that

The iOS device, is also used to receive the second audio, video data packet, obtains the in the second audio, video data packet Two video frames and the second audio frame decode second video frame and second audio frame and play in the iOS device.