CN1588914A - Method for transmitting realtime multimedia data on internet - Google Patents

Abstract

This invention puts forward a method for transmitting real-time multimedia data on Internet including the following steps: A NSPD receives news data messages to the buffer region, B If it is full, some old messages are discarded and the region is rearranged, if it is not full, then C The said message is judged if it is a correct NSPD message, if not, it's discarded and the buffer region is recovered, if it's correct, then D. traversing the NSPD SK control block lists to recall the belonged SK E. After recalling the correct SK control, new message are accessed to the message array of the control block waiting for further process.

Description

A kind of method of transmitting real-time multimedia data on the internet

Technical field

The invention belongs to the multimedia software development field, specifically, relate to a kind of method of transmitting real-time multimedia data on the internet.

Background technology

Generally speaking, the server end of multimedia videoconference system all is positioned on the internet, have independent IP address separately, but the client of most of conference systems all is to be positioned at after the NAT device, be connected to Internet by NAT device, such structure has brought a lot of problems for the exploitation of multimedia videoconference system, for example, generally NAT device is configured to not allow outside active to connect, so the server of conference system is difficult to directly, and the user in NAT initiatively sends audio, video data.

In order to break through the restriction of NAT, the way of systems such as H323 is that the configuration NAT device to outside network opening certain port, utilizes NAT device to transmit data to certain main frame of internal network.This way lacks flexibility, and requires to have the administrator right of NAT device, for domestic consumer, is difficult to realize.

Therefore, the way of NSPD is that new protocol layer is installed in the network protocol stack of system, is positioned on inside and outside client of NAT and the server same agreement is installed respectively, has finished the function of passing through of NAT fire compartment wall in this protocol layer.It provides service by the sockets interface of standard to application program, and add new heading for multi-medium data from upper level applications, then the data behind the package are handed to the more system UDP transport layer of low layer, this method meets the Network Layering design principle, and can be applied to very soon in original system.

In sum, task of the present invention is to solve the communication problem that NAT brings to multimedia application, the NSPD character of coupling system agreement also simultaneously, and the safety encipher that can realize multi-medium data and QoS control etc.

The objective of the invention is to solve the communication problem that NAT brings to multimedia application, on the basis of udp protocol, realized a new procotol, can be widely used in various multimedia applications, the multimedia real-time communication technology of the NAT fire compartment wall that jumps is provided.

Summary of the invention

At top top description, the present invention proposes a kind of method of transmitting real-time multimedia data on the internet, and specifically may further comprise the steps: (A) NSPD receives new data message in the buffering area; (B) if buffering area completely then abandon some old message and redistribute; If buffering area less than enter the following step; (C) judge that according to the information of header this message is correct NSPD message, if not correct NSPD message, dropping packets and reclaim buffering area then; If correct NSPD message then enters the following step; (D) tabulation of traversal NSPD socket controll block retrieves the affiliated socket of this message; (E) after retrieving correct socket control, in the message queue of new message access control piece, wait for the further processing of application program.

According to an aspect of the present invention, wherein step (C) further comprises the value of at first checking type of message, if 0x8048001 shows that then this is that NSPD passes through message, otherwise shows that this may be a NSPD virtual port value.

According to another aspect of the present invention, further comprise step (F) inspection socket IO pattern.

According to another aspect of the present invention, if wherein call WSAAsynSelect, then filled the respective field of socket controll block according to the parameter of this API, message is arrived message dilivery to the SOCKET window, application program can receive SOCKET message, and message is returned and taken away to application program.

According to another aspect of the present invention, if wherein call WSAEventSelect, then NSPD is according to the respective field in its parameter filling socket controll block, receive the event handler of network event, application program can receive the network event notice like this, and message is returned and taken away to application program.

According to another aspect of the present invention, block IO, then activate the event handler that blocks IO, and message is returned and taken away to application program if wherein call.

Description of drawings

Fig. 1 shows the relative position of NSPD in network protocol stack;

Fig. 2 has provided the flow process of identification from the client of different internal networks;

Fig. 3 has provided the relation of application program and NSPD SOCKET;

Fig. 4 has provided the NSPD message and has received flow chart;

Embodiment

To following we with reference to the accompanying drawings, embodiments of the invention are described in detail.

NSPD is an independently network layer service, is arranged in the network protocol stack of system, can be employed procedure Selection and use.Fig. 1 shows the relative position of NSPD in network protocol stack, and according to the layering principle of procotol, it is between UDP transport layer and application layer.The centre position that is in network protocol stack of NSPD, it upwards provides one group of calling interface, for application program provides service, and can admit the concurrent execution of a plurality of application programs simultaneously, downwards, NSPD depends on the UDP transport layer module of system, finishes actual exchanges data on network, and in inside the function of integrated passing through NAT fire compartment wall, transparent for the application program on upper strata.

The access system network layer that the api interface that application program provides by NSPD is indirect, the function of these interfaces should be improved and be easy to use as far as possible.Can reduce following a few class according to their functions separately.

1, creates and cancels the NSPD socket

When application program required to use NSPD, NSPD should be it sign that overall situation is unique is provided, and might as well be referred to as " the NSPD socket ".Similar with the Berkeley socket, application program is finished all network service according to the NSPD socket, and an application program can be created a plurality of NSPD sockets, finishes network operation separately respectively, and each other can be not influential.Based on socket, NSPD provides other API for application program.

Each socket is in the inner corresponding related data structure of NSPD, and we are referred to as socket controll block, have suffered all information relevant with socket of having preserved in controll block.Create from application program the life cycle of socket, cancels up to the program of being employed.NSPD safeguards the information of these sockets, takes place to cancel corresponding socket controll block voluntarily when unusual in application program.

Application program sends or receives packet by corresponding interface by NSPD.During transmission, NSPD is the new field of COM Continuation of Message, does as a wholely in transmission over networks with message content, and during reception, NSPD delivers message content to correct socket according to the content of added field.

Web application obtains data by the IO pattern, and this class method of great majority all has following two kinds of different states in the process that receives message:

1. wait for, up to receiving data

2. message is copied to application program from network layer

In this manner, for putting on the receiving course of NSPD socket, the first step waits for that normally message reaches NSPD from network, after message arrives, be received in the meshwork buffering district of NSPD, in second step, it copied to the buffering area of application program from the buffering area of NSPD.In order to raise the efficiency, NSPD should reduce the number of times that message duplicates as far as possible.

Based on these states, NSPD provides some basic IO patterns, can satisfy the demand of most of application programs.

(1) blocking model

During API that the socket call message receives, pass on the signal of wishing to receive message, be in blocked state then always to NSPD.After network data reached the virtual port of this socket from NSPD, API Calls was returned, and application program obtains data from the buffering area of NSPD.

(2) signal drive pattern

Before preparing to receive message, socket need set message and receive routine, and can return immediately, continues other operation.After this, whenever this socket of data arrives is arranged, NSPD can send corresponding signal to application program, calls the message that sets previously automatically and receives routine.In routine, application program copies to data in the buffering area of oneself from the buffering area of NSPD.

Application program has randomness and unpredictability to calling of these interfaces, therefore NSPD is in one group of mechanism relevant with application program of internal maintenance, to guarantee stability and the robustness of NSPD, these mechanism comprise virtual port mapping, Firewall Traversing mechanism and buffer management etc.

In order to reduce taking of system resource, NSPD is designed to only take a udp port, and the NSPD module on the different main frames is all appointed and used same port.So, when a plurality of application programs are used NSPD to send and accepted data, just need do further differentiation to the destination address of message, we have introduced the notion of virtual port in NSPD for this reason.Virtual port is 16 positive integers, and each NSPD socket all is bound on the unique virtual port of overall situation.When sending message, NSPD appends to its value in each message, as the part of UDP message bag in transmission over networks, when receiving message, NSPD is according to the value of virtual port, determine recipient's socket, and, deliver correct signal and give the application program of creating socket according to the IO pattern of this socket.

The strict differentiation of NSPD is positioned at the inside and outside two kinds of different network environments of NAT fire compartment wall, internally is referred to as the NSPD client, externally is referred to as the NSPD server.Client is positioned at internal network, use Intranet IP address, and server is positioned at external network, and they are intercepted by NAT device.

Utilize the principle of NAT, client needs initiatively to send some data toward server, purpose is to set up the mapping relations of NAT address net address in the client on NAT device, in case this mapping relations are established, server just can be transmitted data to client by corresponding port on the NAT device.

Fig. 3 has provided the flow chart of automatic identification from the client of different internal networks, according to the difference of message present position, flow process is divided into three steps.

1. client is to NAT

Client socket is noted the IP address and the virtual port of this machine, and it is mail to server.Do not pass through before the NAT device local address that source address in the UDP head of message and source port are client, promptly interior net address.

2. through NAT device

NAT device is according to the rule of self, and the source address in the UDP head of message and source port are replaced with the address and the port of the Internet registration that NAT device has and source address and source port before the record modification.

3.NAT to server

Message arrives after the server, two addresses of server record: the address in the UDP head, i.e. NAT address; The client address that writes down in the message content (comprising port), promptly interior net address.These two addresses are kept in the address mapping table of server end.

Client that is positioned at internal network of each correspondence of the address mapping table of NSPD server, when the application program on the external network sends message by NSPD toward Intranet, only need then message be sent to the NAT address and get final product according to interior net address and anti-NAT address and the port found of port.

Below network message is arrived at NSPD buffer management afterwards and describe, it has following characteristics:

High efficiency: NSPD is the multi-media network application programming, and it is big that this class method has network traffics, real-time is required characteristics such as height, so NSPD should reduce the number of copy times of message and the delay that buffering is brought as far as possible, raises the efficiency.

Stable: NSPD has preserved certain information for each application program, even for it has cushioned a large amount of network messages, takes place to have little time to reclaim these resources probably when unusual in application program.NSPD need prevent the resource leakage that this situation causes, guarantees the disposal ability of self.

Fig. 3 has provided the design according to Windows SPI, application call WSASocket function, and the address family that Winsock provides can be according to function call the time, and Socket type and protocol parameter etc. decides needs to load for which ISP.For NSPD,, just can load the NSPD ISP as long as the corresponding protocols parameter is provided.Usually application program adopts following mode to create the NSPD socket:

WSAPROTOCOL_INFO?info；

GetNSPDProviders (﹠amp; Info); // obtain the NSPD protocol parameter

sock＝WSASocket(AF_INET，SOCK_DGRAM，0，

&info，

0，0)；

After having created the NSPD socket, in fact application program has selected use NSPD ISP.The NSPD ISP exists with the form of DLL, and this DLL has imported the single function clauses and subclauses of a WSPStartup by name.In application program (NSPD client computer) when calling WSPStartup, just send table to open other NSPD function by a function that is taken as the parameter delivery, the NSPD ISP is made up of these functions, and the corresponding relation of they and Winsock API is as shown in the table:

????Winsock?API	NSPD ISP API
		????WSASocket	????WSPSocket
????WSARecvFrom/recvfrom	????WSPRecvFrom
		????WSASendto/sendto	????WSPSendto
????WSABind/bind	????WSPBind
		????WSACloseSocket/closesocket	????WSPCloseSocket
????WSAAsynSelect	????WSPAsyncSelect
		????setsockopt	????WSPSetSockOpt
????getsockopt	????WSPGetSockOpt
		????WSAIoctl	????WSPIoctl
????WSAConnect/connect	????WSPConnect
		????WSASend/send	????WSPSend
????Getsockname	????WSPGetSockName
		????WSAEventSelect	????WSPEventSelect

Application program is created after the NSPD socket, all can be mapped to corresponding API among the NSPD ISP on the form left side with the corresponding Winsock API Calls of this socket, for example, finally can finish the bind operation of NSPD socket by the WSPBind function among the NSPD ISP.

In fact the NSPD ISP has taken over application program to the calling of Winsock API, and portion safeguards one group of NSPD socket controll block within it.Each NSPD socket controll block has kept a large amount of information relevant with the NSPD socket corresponding to the some NSPD sockets in the application program in controll block, following table has been listed part key message wherein:

Type	Field name	Implication
			????HANDLE	????m_socket	The SOCKET handle number
????DWORD	????m_error	Socket Errno value
			????DWORD	????m_pid	Create the process ID of socket
????HWND	????m_wnd	With the WSAAsynSelect corresponding data types
			????UINT	????m_msg
????long	????m_event
			????HANDLE	????m_event_sel	The handle number corresponding with WSAEventSelect
????UINT	????m_packets_ ????head	The message queue head
			????UINT	????m_packets_t ????ail	The message queue tail

According to the table in the relevant field of socket IO pattern, NSPD has realized three kinds of different Winsock IO patterns, i.e. WSAAsynSelect, WSAEventSelect and Blocked IO etc.These three kinds of patterns can satisfy the requirement of major applications program basically.

As shown in Figure 3, the size of the socket controll block tabulation that NSPD safeguards is three, have four messages to wait to receive in the message queue of certain socket of creating in the application program, and current in the NSPD message buffer six messages are arranged, the ownership of other two messages does not mark in the drawings.

Fig. 2 has provided the NSPD module and has received message at 8848 ports of UDP, and when message arrived this port, the message that begins NSPD immediately received and handling process.

At first, NSPD attempts from buffering area allocation space and deposits new data message, and message is directly received in the middle of the shared drive, so, application program just can directly be obtained message from shared drive, reduced the number of copy times of message, has improved the efficient of system.Distribute at buffering area under the situation of failure, NSPD can check message buffer, reclaims a part of buffering area according to rule.In the message buffer of NSPD, message carries out buffer memory according to following form:

Type	Field	Implication
			????UINT	????m_next_packet	Next message in the message queue
????UINT	????m_len	Message length
			????SOCKET	????m_sock	The socket that message belonged to
????DWORD	????m_timestamp	The timestamp that message arrives

????sockaddr_ ????in	????m_addr	The source address of message
			????NSP_PA ????CKET	????m_pkt	The NSPD message

Accordingly, the head of NSPD message is defined as follows:

Type	Field	Implication
			????UINT	????m_type	The NSPD virtual port
????UINT	????m_len	Message length

Message among the NSPD is divided into two types, and a kind of is common NSPD data message, and is another kind of for NSPD passes through message, is used for setting up data link from the inner passing through NAT of NAT fire compartment wall, and the formal definition of passing through message is as follows:

Type	Field	Implication
			????UINT	????m_type	Type of message is worth and is 0x8048001
????UINT	????m_len	Message length
			????UINT	????m_ip	Local IP
????UINT	????m_port	The NSPD virtual port of this socket

In sum, two kinds of messages integrate, and adopt C language definition to be:

　　          struct nsppkt{　　             UINT m_typ；　　              int m_len；　　            union{　　               char M_Dat[NSPDATALEN]；　　               struct tag_m_addrPair{　　                     UINT m_ip；　　                     UINT m_port；　　                   }m_addr；　　                }；　　             }；

Message is received after the buffering area, just can judge that this message has been correct NSPD message according to the information of header.At first check the value of type of message, if 0x8048001, show that then this is that NSPD passes through message, just represent that otherwise this may be the virtual port value of NSPD, further, by the tabulation of traversal NSPD socket controll block, just can retrieve the affiliated socket of this message or conclude that this message is an invalid packet.

After retrieving correct socket controll block, in the message queue of new message meeting access control piece, wait for the further processing of application program.

The front is mentioned, and NSPD supports three kinds of different Winsock IO models, is respectively WSAAsynSelect, WSAEventSelect and Blocked IO pattern.NSPD can handle respectively according to the IO pattern of this socket after being put into message in the message tabulation in the socket controll block, and notification application is taken message away from formation.Introduce respectively below:

●WSAAsynSelect

When application call WSAAsynSelect, NSPD has filled the respective field of socket controll block according to the parameter of this API, and they are the window handle that receives SOCKET message, SOCKET message value etc.Therefore the windows messaging that only needs to deliver registration is to correct window handle, and application program can will receive message by the recvfrom function.

●WSAEventSelect

Same, when application call WSAEventSelect, NSPD also can fill respective field in socket controll block according to its parameter, as the event handler that receives network event etc.NSPD can place signal condition with this incident, and so, application program just can be returned from functions such as WSAWaitforSingleObject, receives message by API.

●BlockedIO

NSPD makes the recvfrom function that is in blocked state return application program, directly with datagram

Literary composition returns to application program.

What may be obvious that for the person of ordinary skill of the art draws other advantages and modification.Therefore, the present invention with wider aspect is not limited to shown and described specifying and exemplary embodiment here.Therefore, under situation about not breaking away from, can make various modifications to it by the spirit and scope of claim and the defined general inventive concept of equivalents thereof subsequently.

Claims (6)

1, a kind of method of transmitting real-time multimedia data on the internet comprises:

(A) NSPD receives new data message in the buffering area;

(B) if buffering area completely then abandon some old message and redistribute; If buffering area less than enter the following step;

(C) judge that according to the information of header this message is correct NSPD message, if not correct NSPD message, dropping packets and reclaim buffering area then; If correct NSPD message then enters the following step;

(D) tabulation of traversal NSPD socket controll block retrieves the affiliated socket of this message;

(E) after retrieving correct socket control, in the message queue of new message access control piece, wait for the further processing of application program.

2, according to the process of claim 1 wherein that step (C) further comprises the value of at first checking type of message,, otherwise show that this may be a NSPD virtual port value if 0x8048001 shows that then this is that NSPD passes through message.

3,, further comprise step (F) inspection socket IO pattern according to the method for claim 1.

4, according to the method for claim 3, if wherein call WSAAsynSelect, then filled the respective field of socket controll block according to the parameter of this API, message is arrived message dilivery to the SOCKET window, application program can receive SOCKET message, and message is returned and taken away to application program.

5, according to the method for claim 3, if wherein call WSAEventSelect, then NSPD is according to the respective field in its parameter filling socket controll block, receive the event handler of network event, application program can receive the network event notice like this, and message is returned and taken away to application program.

6, according to the method for claim 3, block IO, then activate the event handler that blocks IO, and message is returned and taken away to application program if wherein call.

Images