CN101815065A

CN101815065A - Long-distance medical treatment real-time information communicating and exchanging method based on IPv6 network SIP (Session Initiation Protocol)

Info

Publication number: CN101815065A
Application number: CN201010034389A
Authority: CN
Inventors: 黄永峰; 柏森; 王方超; 胡萍; 周贤伟
Original assignee: Tsinghua University
Current assignee: Tsinghua University
Priority date: 2010-01-21
Filing date: 2010-01-21
Publication date: 2010-08-25
Anticipated expiration: 2030-01-21
Also published as: CN101815065B

Abstract

The invention relates to a long-distance medical treatment real-time information communicating and exchanging method based on an IPv6 network SIP (Session Initiation Protocol), belonging to the technical field of long-distance medical treatment communication. The long-distance medical treatment real-time information communicating and exchanging method is characterized in that each client comprises an SIP signalling processing module, an audio-video transmission module, an instant short message transmission module, a file transmission module, an electronic whiteboard information transmission module and a medical treatment curve transmission module, wherein the file transmission module, the electronic whiteboard transmission module and the medical curve transmission module adopt user-defined sending and responding messages, and the other modules adopt SIP protocol signalling standard messages. In addition, an SIP signalling control thread is arranged at each client, and the client realizes the communication with other clients through the SIP signaling processing module and a proxy server. The invention can meet the requirement of long-distance medical treatment real-time communication in an IPv4 network and an IPv6 network, and has the advantages of simple realization, good compatibility and easy expanding.

Description

Tele-medicine real-time information interaction method based on IPv6 net Session Initiation Protocol

Technical field:

Belong to the multimedia realtime communication field

Background technology:

Tele-medicine is a kind of new medical model, and it is the product that modern medicine technology, computer networking technology and modern communications technology combine.So-called tele-medicine is meant that generally carrying out bidirectional information by network technology between remote doctor of asking and doctor transmits, and finishes the processes such as enforcement of information search, diagnosis and the medical scheme of asking the doctor.

The tele-medicine biggest advantage be resource-sharing high efficiency, to emergency can emergency treatment ageing, the economy of saving hospital and patient's spending.If can provide tele-medicine by broadband network or telephone line etc., the time guarantee not only can be provided for patient's treatment, reduce the patients ' expenses spending, can also improve the popularity of doctor and hospital.

Network multimedia technology is to realize large scale network key issue in application technology such as tele-medicine, Web conference, interactive mode teaching, game on line, and high speed internet is a basic platform of realizing these application.Multi-medium data transmission and play-back technology are that multidisciplinary, multi-field fusions such as computer technology, the network communications technology, multimedia technology intersect the network application technology that is produced, and are widely used in that multimedia is given a news briefing, fields such as online live, the web advertisement, ecommerce, long-distance education and tele-medicine.

The network communications technology in the tele-medicine is that the development ratio is very fast, also the technology of more complicated.Early stage tele-medicine is not exactly because the communication technology has to access and develops fast and influenced the development of tele-medicine technology.Nowadays, Internet technology has obtained development at full speed, and particularly the IPv6 network is constantly universal, and the platform of development is provided for tele-medicine.

In the real-time information interaction of tele-medicine, the technology that relates generally to has: the transmission of session signaling, and the audio frequency and video real time communication, instant short message sends, case history file transfer and real time medical data (electronic whiteboard and medical curve) transmission.Wherein, session signaling uses Session Initiation Protocol to realize, the RTP/RTCP protocol transmission is used in audio frequency and video communication, and instant short message is carried in the MESSAGE message of SIP signaling and sends, and case history file transfer and real time medical data are with the transmission of packing of self-designed data format.

The Session Initiation Protocol that is used for session signaling transmission is an important techniques wherein.SIP (Session InitiationProtocol) agreement is the ietf standard of releasing calendar year 2001 (RFC3261), is used for foundation on the IP data network, change and ending multimedia session.Expansion along with the standard of SIP can realize multiple business, as IP phone, multimedia conferencing, remote white boarding, instant message etc., and can simplify connection to IP Sec VPN.And SIP can also be easily lumps together the service groups of multimedia service and other types, as directory information, web browsing, locate and attend as a nonvoting delegate service, searching and tracking etc.Because SIP is an application layer protocol, it is relatively independent, and the participant of any kind can be coupled together by SIP on the IP network.For example, SIP can provide the seamless service function between fixing and the mobile network, and this is to realize the fixing key element that merges with mobile service, and, this function along with the 3G mobile network all over the world deployment and become more and more important.

SIP has been the core control protocol of next generation network by generally acknowledging, its function is constantly being expanded, on the basis of SIP basic standard document RFC3261, IETF has released many SIP extension standards, for example RFC3262 (reliability of regulation Temporary Response), RFC3263 (having determined the locating rule of PROXY), RFC3264 (having proposed proposal and answer model), RFC3265 (proposing the event notice rule) and RFC3248 (proposing the instant message rule) etc.

Because Session Initiation Protocol has the characteristics such as simple, compatible good and autgmentability is strong that realize, so, utilize Session Initiation Protocol to carry out the Telemedicine System exploitation and have very big advantage, promptly can be compatible mutually with other software systems with SIP exploitation, can guarantee the upgrading that system is later and the facility of function expansion again.

Based on above-mentioned consideration, the present invention proposes a kind of novel remote medical treatment real time information trade-to product, this product is a kind of medical software terminal based on IPv6 network and Session Initiation Protocol exploitation, in order to satisfy present pandemic IPv4 network application, this product has been realized the function application under the IPv4 network simultaneously.The design philosophy of this product is to carry out negotiation, foundation, modification and the termination of session by Session Initiation Protocol, utilizes technology such as RTP/RTCP to carry out the mutual of multimedia real time information.

Summary of the invention:

The present invention is for a medical data interaction platform of going to a doctor in real time is provided between the strange land medical personnel that give wide apart or doctor and the doctor, not only provide high-quality medical treatment and nursing to being in remote area, save the treatment time for critically ill patient especially, improved treatment efficient.The present invention can provide the mutual of real-time tele-medicine data in the IPv6 high speed internet, medical data wherein comprises real-time audio and video data, medical parameter data (body temperature, heartbeat, pulse, blood pressure etc.), short message and file data in real time.

The present invention can satisfy the tele-medicine real time communication in IPv4 and the IPv6 network, has the advantage that realizes simple, well compatible and easy expansion.

Described method realizes between each client and acting server successively according to the following steps: step (1) client initialization

Set up with lower module in client: Session initiation protocol SIP signaling processing module, audio frequency and video transport module, instant short message transmission module, document transmission module, electronic whiteboard transport module and medical curve transport module, wherein:

The Session initiation protocol SIP signaling processing module, be provided with following sip message: message succeeds in registration, registration failure message, the 200OK response message, the ACK acknowledge message, INVITE audio frequency incoming call message, RE_INVITE video incoming call message, EXOSIP_MESSAGE_NEW message, wherein it comprises instant short message, file sends message, electronic whiteboard message and medical curve message, send message in order to distinguish file, electronic whiteboard message and medical curve message, with the content of text formal definition in the MESSAGE message body is " type of message/message entity " form, the type of message of "/" front is used for the identification message type, wherein instant short message identifies with " MESSAGE ", send a request message usefulness " FILESEND " sign of file, file sends to agree to receive to reply uses " FLSD_BACK " sign, the electronic whiteboard request message is used " WHITEBOARD ", electronic whiteboard is agreed to receive answer and is identified with " WHBD_BACK ", the medical treatment curve identifies with " MEDICALCURVE ", the medical treatment curve agrees to receive answer " MEDCUR_BACK " sign, when being designated " MESSAGE " message, the message entity of "/" back is a content of short message, be designated " FILESEND ", " WHITEBOARD ", or the message entity of "/" back is filled in this machine IP address when " MEDICALCURVE ", also has conversation end message EXOSIP_CALL_CLOSED, port 5070 is used in all signaling transmission

The audio frequency and video transport module is divided into coding and sends part and receive decoded portion, wherein:

Coding sends part, by microphone or camera collection to original audio, video data mail to the recipient, wherein, original audio data is packaged into the RTP bag later on through protocol code G.711, G.723 or G.729 and sends, original video data is packaged into the RTP bag later on through agreement H.263 or H.264 and sends

Receive decoded portion, after the RTP bag decoding that receives, be sent to audio player, audio frequency and video signal post uses port to be followed successively by 15004 and 15005 ports,

Instant short message transmission module is divided into instant short message and sends part and instant short message receiving receiving portions, wherein:

Instant short message sends part, the literal of input equipment input is encapsulated in the described MESSAGE message send,

Instant short message receiving receiving portions is resolved message body and is obtained instant short message from the described MESSAGE message that receives, this instant short message uses the port 5070 of described SIP signaling,

Document transmission module is divided into file and sends part and file receiving unit, wherein:

File sends part, after having determined to send the file path of file, file data is divided into file header information and document content information, send at twice,

The file receiving unit receives file header information and document content information respectively by the file transfer channel, uses port 8023,

The electronic whiteboard transport module is divided into electronic whiteboard information and sends part and electronic whiteboard message pick-up part, wherein:

Electronic whiteboard information sends part, be input as mouse position or push button signalling and be output as image or literal on this electronic whiteboard, send after then described image or literal being packaged into self-defining data structure, the struct structure type that this structure is used in the c++ language is described, content comprises curve type and puts location coordinate information

Electronic whiteboard message pick-up part behind the packet decompress(ion) that receives, shows that with icon or lteral data form it is 5557 that port is used in electronic whiteboard communication again on electronic whiteboard,

Medical treatment curve transport module is divided into medical curve and sends part and medical curve receiving unit, wherein:

The medical treatment curve sends part, the medical data that collects by the packing of self-defining data structure after, show in this locality with curve form, simultaneously described medical data is sent, the struct structure type that this structure is used in the c++ language is described, content comprises the paintbrush attribute and puts location coordinate information

Medical treatment curve receiving unit after the medical data that receives unpacked, shows that at medical curve window the port of transmission use is 5559,

Client has also set up following SIP signaling control thread, audio collection thread, audio coding transmission thread, video acquisition thread, video coding transmission thread, audio frequency reception decoding line journey, voice playing thread, video reception decoding thread, video playback thread, medical curve to send thread, medical curve receiving thread, electronic whiteboard transmission thread, electronic whiteboard receiving thread, file transmission thread and file receiving thread, and step (2) client realizes the mutual transmission of tele-medicine real time information successively according to the following steps:

Step (2.1) client is registered to described acting server:

Step (2.1.1) client sends request registration message REGESTER to described acting server,

Step (2.1.2) client is received the successful respond message 200OK that described acting server sends,

When step (2.2) realizes that by described acting server the audio frequency and video module communicates when client, be from the following communication pattern of choosing any one kind of them pattern or the passive receiving mode of initiatively initiating,

The communication pattern of initiatively initiating under audio conditions carries out according to the following steps:

At first, the sip address sign of input the other side client is pressed dialing button, initiates call request, sends the INVITE request message,

Secondly, when the 180 provisional response message of successively receiving the other side successively and the final response message of 200OK, send acknowledge message ACK, start described audio frequency simultaneously and receive decoding line journey, audio coding transmission thread, voice playing thread and audio collection thread,

Passive received communication pattern under audio conditions is carried out according to the following steps:

The first step is created a watcher thread, monitors external SIP communication request instruction,

In second step, when the described INVITE request that listens to that the other side's client sends, local ringring prompting client has to call out and enters, and simultaneously, replys 180 provisional response message,

The 3rd step when client accepts request, sent the 200OK response message to described the other side's client, till the other side sends acknowledge message ACK,

The 4th step after receiving the other side's client acknowledge message ACK, started described audio frequency simultaneously and receives decoding line journey, audio coding transmission thread, voice playing thread and audio collection thread,

Under the video condition, initiatively initiate communication pattern and carry out according to the following steps:

At first, input the other side sip address sign is pressed dialing button again, sends the RE_INVITE request message,

Next when the 180 provisional response message of successively receiving the other side successively and the final response message of 200OK, sends acknowledge message ACK, starts video reception decoding thread, coding transmission thread, video playback thread and video acquisition thread simultaneously,

Under the video condition, passive receiving mode carries out according to the following steps:

In second step, when listening to the request message RE_INVITE of the other side's client initiation, local ringring prompting client has calling to enter, and replys 180 provisional response message simultaneously,

The 3rd step, accept communication request when client, press when answering button, send the 200OK response message to described the other side's client, till receiving that the other side confirms to ask ACK,

The 4th step when receiving the other side's client acknowledge message ACK, started described video reception decoding thread, video coding transmission thread, video playback thread and video acquisition thread simultaneously,

When step (2.3) realizes instant short message communication when client by described acting server, carry out according to the following steps:

Utilize described SIP signaling processing module that the instant text message of needs transmission is used the XML language formatization, send to the other side's client by described MESSAGE message again, the message format of instant short message is revised as " MESSAGE/ message entity ", " message entity " is described instant content of short message

After the other side's client is received sip message MESSAGE message, resolve the XML text that message body includes, obtain described instant short message,

When step (2.4) realizes that by described acting server file sends communication when client, carry out according to the following steps:

Client is used the socket Socket of TCP pattern, sends thread by described file and sends file, and the other side's client uses the file receiving thread to receive described file,

Client will use the file FILESEND that sends a request message to send file, client is used file transmission response message FLSD_BACK to agree to receive and is replied, file sends a request message formal definition for " FILESEND/ message entity ", " message entity " represents this machine IP address, agree that receiving the response message formal definition is " FLSD_BACK/ message entity ", " message entity " represents acknowledged client terminal IP address

When step (2.5) realizes the electronic whiteboard message communicating when client by described acting server, carry out according to the following steps:

Client uses electronic whiteboard request message WHITEBOARD to initiate the electronic whiteboard communication request, the other side's client uses electronic whiteboard communication to agree response message WHBD_BACK, the electronic whiteboard request message format is defined as " WHITEBOARD/ message entity ", " message entity " represents this machine IP address, agree that receiving the response message formal definition is " WHBD_BACK/ message entity ", " message entity " represents acknowledged client terminal IP address

When step (2.6) realizes that by described acting server medical curve is communicated by letter when client, carry out according to the following steps:

Client uses medical curve communication request message " MEDICALCURVE " to initiate medical curve communication request, the other side's client agrees that receiving response message is " MEDCUR_BACK ", medical treatment curve request message format is defined as " MEDICALCURVE/ message entity ", " message entity " represents this machine IP address, agree that receiving the response message formal definition is " MEDCUR_BACK/ message entity ", " message entity " represents acknowledged client terminal IP address.

Description of drawings:

Fig. 1. system platform figure.

Fig. 2. terminal overall procedure block diagram.

Fig. 3. the client terminal design drawing.

Fig. 4. each thread communication figure of client terminal.

Fig. 5. the sub-thread communication figure of audio frequency and video.

Fig. 6. the voice communication procedure chart.

Fig. 7. electronic whiteboard is realized basic framework.

Fig. 8. electronic whiteboard thread graph of a relation.

Fig. 9. each sub-thread module data flow graph.

Figure 10. main thread operation workflow figure.

Figure 11. real-time curve drafting module working mechanism.

Figure 12. real-time curve drawing process schematic diagram.

Embodiment:

Telemedicine System adopts traditional C/S architecture design.The design of system mainly comprises the configuration of server and the exploitation of software client terminal.Server mainly is responsible for the mutual of SIP signaling, setting up when session, also is that the SIP signaling is called out when finishing startup medical data transmission system the mutual support that no longer needs server of data, and being to use the end-to-end transmission of peer-to-peer, the entire system structure is as shown in Figure 1.

1. terminal general frame

The function that the tele-medicine terminal software is realized mainly includes audio frequency and video communication, instant message transmission, file transmission, electronic whiteboard and medical curve.Carry out tele-medicine, the sensation that will " be close at hand " for the people exactly, wherein audio frequency and video communication is essential; When network environment when not being fine, we wish the communication support that can provide minimum, and instant message sends because the data volume that takies is less, but reception and registration that can the be clear and definite individual meaning, thereby need comprise the instant message function; Exchange the vital documents such as electronic health record that all unavoidably will exchange patient between interrogation between patient and the doctor or doctor and the doctor, can carry out that file sends this moment is a very important function; If both sides need check a certain pictures jointly,, at this moment show and the interchange of carrying out both sides is necessary with electronic whiteboard such as patient's X-ray picture; The medical treatment curve can be real-time some physiological parameters of demonstration patient such as electroencephalogram (EEG) curve, electrocardiogram curve etc., the running of whole software terminal is as shown in Figure 2.

Realize above function, hierarchical design thought is adopted in the design of tele-medicine software terminal, and software is divided into user interface layer, interface layer, functional layer and transport layer.Wherein, it is in functional layer that the function of software realizes, mainly comprises two modules: SIP signaling processing module and medical data processing module.

As shown in Figure 3, user interface layer is the superiors that are positioned at whole application program, and graphical interfaces is provided, and determines the framework of software.Interface layer, it calls some interface functions, has shielded the details of lower module, and some simple interfaces are provided, and is convenient to that the call back function of control calls in the user interface layer, realizes the butt joint of user interface with functional layer.Functional layer realizes the parsing and the generation of SIP signaling, is responsible for finishing the processing of medical data, is the main part of whole software, is the key modules that each function realizes.The TCP/UDP transport layer is responsible for realizing the bottom socket transmission of medical data.

2. each functional module realizes

Relatively independent between each function of the design, use sub-thread to realize respectively.When program start, at first move SIP signaling control thread and monitor sip request message.The used thread of other modules has only when corresponding module is set up service, and sub-thread just can be set up, and is to use same the sub-thread with SIP signaling control module except instant short message sends in each module, and other modules all need to set up oneself independently thread.Communication between each sub-thread is as shown in Figure 4:

(1) sip message processing module

The sip message module is main foundation, maintenance and the release of being responsible for session in the middle of whole tele-medicine conversation procedure.Before the tele-medicine session was set up, client was at first initiated registration (Register) to registrar, sent REGESTER message, and server will be checked customer data base, and gave the location server preservation IP address and user name.

Registration is just finished and can be called out and talk with.The sip message processing module has two critical functions, and the one, when making a call, carry out the mutual of SIP signaling; Whether the 2nd, monitoring constantly has sip request message to arrive, and the monitoring of sip request message is to realize by setting up an independent SIP signaling control thread.

Server mainly is to provide service by the SER server, and the client design exploitation is an emphasis.The SIP Signalling exchange of client terminal comprises in this product: the signaling in the standard agreement is used and oneself is designed signaling and use two parts.Use REGESTER message to register during the user end to server registration, the session processing signaling between client and the client is as shown in table 1 below:

Table 1: mutual signaling between the client

Service is divided	Calling party's message	Callee's message
Service is divided	Calling party's message	Callee's message	Audio session	??INVITE、ACK	??180、200OK
Video session	??RE_INVITE、ACK	??180、200OK	Audio session	??INVITE、ACK	??180、200OK
Video session	??RE_INVITE、ACK	??180、200OK	Instant message	??MESSAGE	??MESSAGE
File sends	??FILESEND	??FLSD_BACK	Instant message	??MESSAGE	??MESSAGE
File sends	??FILESEND	??FLSD_BACK	Electronic whiteboard	??WHITEBOARD	??WHBD_BACK
The medical treatment curve	??MEDICALCURVE	??MEDCUR_BACK	Electronic whiteboard	??WHITEBOARD	??WHBD_BACK

Wherein, the signaling of using in audio frequency and video session and the instant message is a specified standard message in the Session Initiation Protocol standard, and signaling is self-designed signaling in file transmission, electronic whiteboard and the medical curve.

■ is to server registration

Client-server

|--------------------REGESTER------------------------＞|

|＜-------------------200OK------------------------------|

Audio frequency and video conversation between the ■ client

Server 1... server 2

proxy?????proxy

UC1........................................UC2

|?????????????????|?????????|???????????????|

|INVITE?F1????????|??????????|???????????????|

|---------------＞|INVITE?F2??|??????????????|

|?????????????????|---------------＞|INVITE??F4|

|?????????????????|?????????????????|-----------------＞|

|?????????????????|?????????????????|200O?K??F9|

|?????????????????|200OK?F10????????|＜----------------|

|200OK??F11???????|＜----------------|?????????|

|＜---------------|?????????????????|??????????|

|?????????????????ACK??F12?????????????????????|

|---------------------------------------------------------------＞|

|????????????????Audio?Media?Session???????????|

|＜＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＞|

|?????????????????|?????????????????|??????????|

|RE_INVITE?F13????|?????????????????|??????????|

|----------------------＞|RE_INVITE??F14|??????|

|?????????????????|----------------------＞|RE_INVITE?F15|

|?????????????????|?????????????????|----------------------＞

|?????????????????|?????????????????|200OK??F16|

|?????????????????|200OK??F17???????|＜---------------------|

|200OK??F18|＜----------------------|??????????|

|＜----------------------|??????????|??????????|

|??????????????????ACK??F19????????????????????|

|-----------------------------------------------------------------------＞|

|?????????????????????????Video?Media?Session?????????????????????????????|

|＜＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＞|

|?????????????????????????BYEF20??????????????????????????????????????????|

|＜-----------------------------------------------------------------------|

|?????????????????????????200OK?F21???????????????????????????????????????|

|-----------------------------------------------------------------------＞|

Short message communication between the ■ client

UC?1(Sender)????????????????????????????????????UC?2(Receiver)

|-------------------MESSAGE----------------------------＞|

|＜------------------200OK----------------------------------|

Annotate: for the message transmission of extendfile transmission, electronic whiteboard, medical curve, XML content of text in the MESSAGE message body is expressed as " type of message/message entity " form, "/" type of message before is used for identifying different message, message entity after "/" is represented different contents in different message, the MESSAGE message format is " a MESSAGE/ message entity ", wherein " message entity " concrete instant short message for sending.

File between the ■ client sends

UC?1(Sender)????????????????????????????????????UC?2(Receiver)

|-------------------FILESEND----------------------------＞|

|＜------------------FLSD_BACK--------------------------|

Annotate: FILESEND message concrete form is " a FILESEND/ message entity ", and wherein " message entity " is file transmit leg client ip address;

FLSD_BACK message concrete form is " a FLSD_BACK/ message entity ", and wherein " message entity " is file recipient client ip address;

Electronic whiteboard session between the ■ client

UC?1????????????????????????????????????????????UC?2

|-------------------WHITEBOARD----------------------＞|

|＜------------------WHBD_BACK------------------------|

Annotate: WHITEBOARD message concrete form is " a WHITEBOARD/ message entity ", and wherein " message entity " is electronic whiteboard session initiator client ip address;

WHBD_BACK message concrete form is " a WHBD_BACK/ message entity ", and wherein " message entity " is electronic whiteboard session recipient client ip address;

Medical curve session between the ■ client

UC?1(Sender)????????????????????????????????UC?2(Receiver)

|-------------------MEDICALCURVE-------------------＞|

|＜------------------MEDCUR_BACK---------------------|

Annotate: MEDICALCURVE message concrete form is " a MEDICALCURVE/ message entity ", and wherein " message entity " is medical curve session initiator client ip address;

MEDCUR_BACK message concrete form is " a MEDCUR_BACK/ message entity ", and wherein " message entity " is medical curve session recipient client ip address;

(2) audio frequency and video processing module

Adopt the multi-thread design thinking, the audio frequency and video transport module distributes 9 threads altogether, and as Fig. 5, each thread function is as follows:

SIP signaling monitoring thread: be responsible for monitoring external sip message.

Audio frequency receives the decoding line journey: be responsible for receiving the RTP packets of audio data and decoding.

Audio coding sends thread: be responsible for sending carrying out the RTP packing behind the audio data coding that collects.

Voice playing thread: decoded RTP packets of audio data is carried out this locality play.

Audio collection thread: utilize microphone to gather voice data, and the data that collect are put into message queue.

Video reception decoding thread: be responsible for receiving the RTP video packets of data and decoding.

Video coding sends thread: be responsible for sending carrying out the RTP packing behind the video data encoding that collects.

Video playback thread: decoded RTP video packets of data is carried out this locality play

Video acquisition thread: utilize the camera collection video data, and carry out this locality demonstration and put into message queue then

Main thread starts, and realizes the setting of some parameters when initialization, as obtains local IP, setting program communication port Port etc.The transmission of audio frequency and video is divided into initiatively initiates the communication two kinds of patterns of communicating by letter with passive acceptance.The flow chart demonstration of the whole process of voice communication is as Fig. 6.

Realize the passive communication pattern (audio frequency is an example) of accepting:

The first step will be monitored external SIP communication request signaling (this program is an eXosip protocol stack signaling) constantly, therefore needs to create a sub-thread of special use (thread 1) and carries out the monitoring of external eXosip signaling, and thread entrance function is made as sip_event_listen.

Second step listened to EXOSIP_CALL_INVITE message, and when promptly the other side initiated the INVITE request, local ringring prompting user had calling to enter, and recovers 180 provisional response message simultaneously in ring.

In the 3rd step, when the user accepts request, promptly the user presses when answering button, just begins to send the final response message of 200OK till receiving the other side ACK acknowledge message to the other side.

The 4th step after receiving the other side ACK acknowledge message, just started audio frequency and receives decoding line journey (thread 2), and thread entrance function is made as os_sound_start_thread; Start audio coding and send thread (thread 3), thread entrance function is made as os_sound_start_out_thread; Start voice playing thread (thread 4); Start audio collection thread (thread 5).

The process of setting up of video communication is similar with the voice communication process, only second step receive for EXOSIP_CALL_REINVITE message, i.e. RE_INVITE message.Receive the ACK acknowledge message and also will start 4 sub-threads later on, be respectively video reception decoding thread (thread 6), entrance function os_video_start_thread, video coding sends thread (thread 7) os_video_start_out_thread, video playback thread (thread 8) and video acquisition thread (thread 9).

Realize initiatively initiating communication pattern (audio frequency is an example):

The first step, the user imports the other side's sip address sign, and presses dialing button and initiate call request, promptly sends the INVITE request message.

Second step, when receiving the other side's 180 provisional response message, point out the other side just in ring in this locality, when receiving the final response message of the other side 200OK once more, send the ACK acknowledge message, start audio frequency simultaneously and receive decoding, audio coding transmission thread and voice playing, 4 threads of audio collection thread, the 4th step in the similar passive receptive pattern of detailed process.

The initiation process of video communication is that INVITE changes RE_INVITE message into audio class seemingly, will set up 4 threads equally.

(3) instant message module

Session Initiation Protocol realizes that the text transmission of instant messaging is not the core content of SIP, but realizes the transmission of instant message by the expansion [9] of Session Initiation Protocol.The sip message of instant message expansion is MESSAGE, the message format of this message is realized according to the standard of Session Initiation Protocol, need the instant text message of transmission to use the XML language formatization, send by MESSAGE and to send to the callee, after the callee receives MESSAGE message, resolve the XML formatted text in the message body, obtain the instant message that calling party is sent.Therefore, the realization of instant message module can be regarded as and realizes in the SIP signaling processing module.

(4) file send module

After the SIP session that file sends is set up, the formal file that sends of beginning.The process of transmitting of file is divided into the transmission of file information head and transmission two parts of file content, newly-built one of transmit leg sends sub-thread during transmission, and newly-built one of recipient receives sub-thread, and transmission is finished, sub-thread is closed, and directly uses the socket transmission of TCP pattern.

This product is the exploitation of carrying out under the windows environment, and therefore, the windowsAPI function is used in obtaining of fileinfo.Whole attribute informations about file, add up to following 9 kinds: the title name of file, the attribute of file (read-only, file is hidden etc.), the creation-time of file, the last access time of file, the last modification time of file, the high-order double word of file size, the low level double word of file size, reservation, reservation.Have only the length of files-designated autograph and file to obtain eaily here by the CFile class, and just powerless for obtaining and being provided with of other several attributes.

All file attributes more than having comprised in the member variable of a data structure WIN32_FIND_DATA who often uses when going to search disk file with FindFirst () and FindNext () function therefore can be by this structure as the means of obtaining and change file attribute.Theing contents are as follows of this structure:

typedef?struct_WIN32_FIND_DATA{

DWORD dwFileAttributes; // file attribute

FILETIME ftCreationTime; // document creation the time

FILETIME ftLastAccessTime; The last access time of // file

FILETIME ftLastWriteTime; // file Last modification the time

DWORD nFileSizeHigh; // file size is high 32

DWORD nFileSizeLow; Low 32 of // file size

DWORD dwReserved0; // system keeps

DWORD dwReserved1; // system keeps

TCHAR cFileName[MAX_PATH]; // long filenames

TCHAR cAltemateFileName[14]; // 8.3 formatted file names

}WIN32_FIND_DATA，*PWIN32_FIND_DATA；

Can deposit association attributes for the treatment of operation file is read in the WIN32_FIND_DATA structure of path searching this document according to current file by FindFirstFile () function:

WIN32_FIND_DATA?ffd；

HANDLE?hFind＝FindFirstFile(″c:\\test.dat″，&fid)；

Can not revise any data in this structure when using this structure by hand, structure can only be as a read-only data, and its all member variable all can be finished by system and fill in.

In order better to preserve the file attribute information that gets access to, corresponding to self-defining SOCKET_STREAM_FILE_INFO data structure of file attribute structure, the attribute information that obtains can be temporary in the middle of this data structure, and SOCKET_STREAM_FILE_INFO thes contents are as follows:

typedef?struct_SOCKET_STREAM_FILE_INFO{

TCHAR szFileTitle[128]; The title name of // file

DWORD dwFileAttributes; The attribute of // file

FILETIME ftCreationTime; The creation-time of // file

FILETIME ftLastAccessTime; The last access time of // file

FILETIME ftLastWriteTime; The last modification time of // file

DWORD nFileSizeHigh; The high-order double word of // file size

DWORD nFileSizeLow; The low level double word of // file size

DWORD dwReserved0; // keep, be 0

DWORD dwReserved1; // keep, be 0

}SOCKET_STREAM_FILE_INFO，*PSOCKET_STREAM_FILE_INFO；

At first using FindFirstFile () function that file attribute is got access in the WIN32_FIND_DATA structure goes, can it be closed with FindClose () afterwards, and standby in the structure objects of self-defined structure SOCKET_STREAM_FILE_INFO the content replication of the relevant document attribute information in the WIN32_FIND_DATA structure.

(5) electronic whiteboard module

Electronic whiteboard software need provide a kind of special software interface, its function has: the both sides that carry out in the remote diagnosis can share a blank simultaneously, the two all can browse to blank information, and on this blank standard shape such as draw line segments, rectangle, ellipse, also irregular curve can be drawn, and the attribute such as brush color, paintbrush thickness of these shapes can be freely selected; The user also can paste patient's medical picture in blank, as x mating plate, CT sheet etc., the both sides that participate in diagnosis can mark on this figure, comprise setting-out and input characters, to impart one's ideas; As long as have a side to open blank in the diagnostic procedure, an other side also ejects its oneself blank interface, not only can see the operation of the other side on blank on this interface, also can oneself paint in the above, stickup, edited image.

The core of this module is a kind of message shared mechanism model.The message shared mechanism of this software design will be formulated a series of application layer protocol, and with the meaning of data in the differentiation packet, as data are character value or coordinate points in wrapping.Share Model structured flowchart under this working method is as shown in Figure 7:

The realization thinking of this module is as follows:

The exploitation of employing multithreading realizes that with the electronic whiteboard function main thread is responsible for the maintenance of software terminal integral body in 3 sub-threads, and main thread is used to produce software interface, and the action of coordinating each sub-thread, uses message informing mechanism with sub-thread communication; But also process user operation of main thread is shown to user related information; Also need at last to utilize main thread to carry out garbage reclamation.The relation of each sub-thread and main thread is as shown in Figure 8:

Between each sub-thread mutually the cooperation, finish different functions respectively, its data flow diagram as shown in Figure 9:

Drawing thread operation graphics module, graphics module is responsible for the coordinate data in the buffer memory is plotted in the painting canvas, and the coordinate data in the buffer memory can be the generation of user's mouse action, also can be the drawing coordinate data of the distance host that receives.

Send the packing of data thread service data, sending module, this module is packed the coordinate data in the buffer memory of graphics module generation, and sends to distance host; This module also is responsible for sending solicited message.The data structure of using in electronic whiteboard is self-defining data structure, and its structure is as follows:

typedef?struct?MSG_BUFFER_WHITE{

LPSTR?operations；

Int Pentype; // selection paintbrush type

Int xPoint; The x coordinate of // picture point

Int yPoint; The y coordinate of // picture point

}MSG_BUFFER_WHITE，*PMSG_BUFFER_WHITE；

Receive the reception of data thread service data, parse module, this module is received the drawing coordinate data that distance host transmits, and these data is unpacked and is saved in the buffer memory call for graphics module; This module also is responsible for receiving solicited message.

The main thread module produces when electronic whiteboard starts, and starts other three sub-threads by this thread again after starting this thread.The major function of main thread is user interface demonstration, promoter thread, send a notification message, receive notification message, garbage reclamation and the resource cleaning of sub-thread to sub-thread.This module is the nucleus module of native system.Its implementation procedure as shown in figure 10.

This thread module is produced by main thread, and major function is that user's mouse action is noted in the mode of coordinate array, and according to different selection (rectangle, prototype, free line) it is plotted on the drawing board.If distance host is agreed to connect, then the coordinate array is stored in the transmission buffer memory.If local host agree to receive data, then will receive storage in the buffer memory in the coordinate array, and be plotted on the drawing board.

Draw and can draw after sub-thread is created, drawing step records mouse action in the coordinate array earlier in two steps, draws on drawing board according to coordinate array and selected plotting mode (rectangle, circle, line etc.) again.

This thread module is produced by main thread, major function is to utilize ICP/IP protocol, mode with socket is packed request msg and draw data that local host produces, and sends to distance host, and wherein the packet of request msg and draw data follows a certain form.

Send and carry out corresponding actions according to the notice of main thread after sub-thread is created.Also created simultaneously the transmission socket when sending sub-thread and being created.

This thread module is produced by main thread, major function is to utilize ICP/IP protocol, the data of sending with the mode receiving remote main frame of socket, and in this locality request msg and the draw data of receiving unpacked, wherein the packet of request msg and draw data follows a certain form.

Receiving sub-thread is created the back and carries out corresponding actions according to the information of the distance host that receives.Also created simultaneously the reception socket when receiving sub-thread and being created.

(6) medical curve module

The working mechanism of real-time curve drafting module such as Figure 11.This module mainly utilizes express network that data are transmitted, and receiving terminal receives data and handles in real time, utilizes real time data to carry out curve plotting and real-time update curve in software interface, finally reaches desired purpose.

As shown in figure 11, in the drawing process of whole real-time curve, relate generally to three courses of work: at first be in of the collection of data acquisition end, send to local system by network to patient's real time data, system utilize again the real-time curve drafting module with curve display in the interface; Meanwhile, local system is analyzed these data and these data is packed, and utilizes express network to send to the system in strange land; In the strange land that is separated by a thousand li, system receives data, principle with same drafting real-time curve, these real-time curves are drawn in the systems soft ware interface of oneself, thereby reach the purpose that both sides' near-synchronous is observed patient's real-time physiological curve, for the co-therapy patient provides strong supplementary means.

The data format that uses in the medical treatment curve plotting packing transmission is the self-defining data structure, and its concrete structure is as follows:

typedef?struct?MSG_BUFFER_CURVE{

LPSTR?operations；

Int Curvetype; // curve type

Int xPoint; The x coordinate of // picture point

Int yPoint; The y coordinate of // picture point

}MSG_BUFFER_CURVE，*PMSG_BUFFER_CURVE；

The real-time curve drafting module is to call realization by the menu function of checking real-time curve in the main interface, so the design of its internal interface mainly is to realize module invokes by call function, thereby realizes the drafting and the demonstration of curve.

The user interface of module is a curve display zone, and the reference axis type is mated automatically according to shown data type in this zone.In the viewing area, by curve uninterruptedly redraw dynamic change from curve to the observer that show because the time interval is very short, In the view of the observer, curve be real-time update not at interval, therefore, doctor's judgement is not had substantial influence.

The drawing process schematic diagram of real-time curve as shown in figure 12.

Step 1: the real-time curve drafting module at first receives packet from network, calls and handles function first the corresponding data point that unpacks and draw;

Step 2: module continues to receive data, and second data point of drawing uses the same method;

Step 3: have regular hour and space interval between preceding two data points, because this is enough little at interval, the method that adopts the picture straight line to connect at 2 can show two front and back contacts between data point very approx.

Circulation repeats above step, the demonstration range that surpasses the curve display zone until data volume, after this receiving data information again, can from the viewing area, wipe data presented point the earliest, other data point moves forward successively, promptly redraws curve, and the data of the up-to-date reception of drawing in the rearmost position of viewing area, until at last, this process just shows as the drafting of real-time curve on macroscopic view with this kind method.

Claims

1. based on the tele-medicine real-time information interaction method of IPv6 net Session Initiation Protocol, it is characterized in that described method realizes successively according to the following steps between each client and acting server:

Step (1) client initialization

The Session initiation protocol SIP signaling processing module, be provided with following sip message: message succeeds in registration, registration failure message, the 200OK response message, the ACK acknowledge message, INVITE audio frequency incoming call message, RE_INVITE video incoming call message, EXOSIP_MESSAGE_NEW message, wherein it comprises instant short message, file sends message, electronic whiteboard message and medical curve message, send message in order to distinguish file, electronic whiteboard message and medical curve message, with the content of text formal definition in the MESSAGE message body is " type of message/message entity " form, "/" front is a type of message, wherein instant short message identifies with " MESSAGE ", send a request message usefulness " FILESEND " sign of file, file sends to agree to receive to reply uses " FLSD_BACK " sign, the electronic whiteboard request message is used " WHITEBOARD ", electronic whiteboard is agreed to receive answer and is identified with " WHBD_BACK ", the medical treatment curve identifies with " MEDICALCURVE ", the medical treatment curve agrees to receive answer " MEDCUR_BACK " sign, when being designated " MESSAGE " message, "/" back message entity is a content of short message, be designated " FILESEND ", " WHITEBOARD ", or this machine IP address is filled in "/" back when " MEDICALCURVE ", also has conversation end message EXOSIP_CALL_CLOSED, port 5070 is used in all signaling transmission

Client has also been set up following thread: SIP signaling control thread, audio collection thread, audio coding transmission thread, video acquisition thread, video coding transmission thread, audio frequency reception decoding line journey, voice playing thread, video reception decoding thread, video playback thread, medical curve send thread, medical curve receiving thread, electronic whiteboard transmission thread, electronic whiteboard receiving thread, file transmission thread and file receiving thread

Step (2) client realizes the mutual transmission of tele-medicine real time information successively according to the following steps:

Step (2.1) client is registered to described acting server: