CN108990169B

CN108990169B - Data transmission and resource sharing method, user terminal and access point equipment

Info

Publication number: CN108990169B
Application number: CN201810662390.7A
Authority: CN
Inventors: 贾鑫
Original assignee: Xian Zhongxing New Software Co Ltd
Current assignee: Xian Zhongxing New Software Co Ltd
Priority date: 2017-09-08
Filing date: 2018-06-25
Publication date: 2022-01-21
Anticipated expiration: 2038-06-25
Also published as: CN108990169A

Abstract

A data transmission method, a resource sharing method, a user terminal and an access point device are provided, wherein the data transmission method comprises the following steps: the first user terminal and the second user terminal exchange the information of respective external equipment, and instruct the external equipment of the terminal to establish data service connection with the external equipment of the second user terminal, and carry out data transmission through the data service connection. The resource sharing method comprises the following steps: the user terminal requests information of the external device to the external device through near field communication, and uses peripheral external hardware resources according to the information of the external device. The application also provides a corresponding user terminal and access point equipment, and the user terminal can make full use of peripheral external hardware resources.

Description

Data transmission and resource sharing method, user terminal and access point equipment

Technical Field

The present invention relates to the field of multimedia communication, and more particularly, to a method for data transmission and resource sharing, a user terminal and an access point device.

Background

Multimedia communication has been a pursuit of goals in the field of communications and is continuing to improve. At present, the mobile phone has the problems of limited data flow, limited multimedia acquisition and playing equipment, limited terminal computing capacity and the like in the aspects of multimedia playing, video call, file transmission and the like, and specifically comprises the following steps:

1. data traffic limiting

In the existing communication system, only information interaction between the mobile terminal and the base station exists, and the local mobile terminal is communicated with the base station only by a transmission main line. On one hand, when the data volume transmitted to the local base station by the local mobile terminal is large, the transmission speed is very low, and the transmission efficiency is reduced; when the data volume to be transmitted by the mobile terminal is larger, the capacity expansion of the base station can only be relied on to improve the network accommodation degree, so that the transmission efficiency of the data is improved, but the capacity expansion of the base station needs to consume great financial and material resources, and the base station cannot expand the capacity infinitely; on the other hand, the cost is much higher than that of Wi-Fi transmission when the data is transmitted through the base station, and the cost paid by a user with large data traffic is higher. A short-distance transmission system Wi-Fi transmission system of the mobile terminal can meet short-distance communication between the mobile terminals, but local area network communication and wide area network communication are isolated, and the two kinds of communication cannot be combined and applied. Short-range communication resources are wasted.

2. Multimedia acquisition and playback device limitations

At present, mobile equipment (UE), especially mobile phones, have popularized a front camera and a large screen, and can implement video calls on the UE. However, due to the hardware limitation of the mobile device and the limitation of the multimedia capability, external devices such as a large screen, an external microphone sound with better sound quality, a high-definition camera and the like cannot be experienced.

3. Limited computing power of terminal

The current mobile device can already support multitask concurrent processing, but is still limited by hardware such as a terminal processor memory, so that the speed of multitask processing is reduced and heat and power consumption are increased.

Disclosure of Invention

The embodiment of the invention provides a data transmission method, which comprises the following steps:

after a first user terminal establishes signaling connection with a second user terminal, the first user terminal sends information of external equipment of the first user terminal to the second user terminal and acquires the information of the external equipment of the second user terminal, wherein the information of the external equipment comprises a communication identification address of the external equipment;

the first user terminal indicates the external equipment of the terminal, and establishes data service connection between the external equipment of the first user terminal and the external equipment of the second user terminal according to the exchanged information of the external equipment of the second user terminal;

and the first user terminal instructs the external equipment of the terminal to carry out data transmission through the data service connection.

The embodiment of the invention also provides a resource sharing method, which comprises the following steps:

a user terminal requests information of an external device to the external device through near field communication;

and the user terminal receives the information of the external equipment returned by the external equipment and uses peripheral external hardware resources according to the information of the external equipment.

The embodiment of the invention provides a resource sharing method, which comprises the following steps:

the external equipment receives an information request sent by the user terminal through near field communication;

the external equipment acquires resource information of peripheral equipment connected with the external equipment, and returns the resource information of the peripheral equipment serving as capability description information of the external equipment to the user terminal so that the user terminal can use resources of the peripheral equipment.

The embodiment of the present invention further provides a user terminal, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the processor implements the processing of any method executed by the user terminal according to the embodiment of the present invention.

The embodiment of the present invention further provides an access point device of a near field communication network, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the processing of any method executed by an external device according to the embodiment of the present invention.

Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the processing of any of the methods according to the embodiments of the present invention.

The scheme of the embodiment of the invention can enable the user terminal to fully utilize peripheral external hardware resources. For example, in some examples, data transmission may be performed using communication resources, multimedia communication may be performed using multimedia device resources, calculation may be performed using processor resources, and so on, so as to achieve one or more effects of improving transmission bandwidth, enhancing user experience, reducing communication cost, and so on, and solve one or more of the problems of limited bandwidth, limited multimedia capability, limited processing capability, and so on of the existing user terminal.

Drawings

FIG. 1 is a diagram of an application environment according to a first embodiment of the present invention;

FIG. 2 is a flow chart of a data transmission method according to an embodiment of the invention;

FIG. 3 is a diagram illustrating the transmission of control signaling between a UE1 and a UE2 according to a first embodiment of the present invention;

FIG. 4 is a schematic diagram of an 802.11 frame format;

FIG. 5 is a diagram illustrating a system architecture for implementing a multimedia call between a UE1 and a UE2 according to an example of the first embodiment of the present invention;

FIG. 6 is a diagram illustrating a hotspot connection option in the first example of the first embodiment of the invention;

FIG. 7 is a flowchart of a resource sharing method at a second user terminal according to an embodiment of the present invention;

FIG. 8 is a flowchart of a resource sharing method of an external device according to an embodiment of the present invention;

fig. 9 is a schematic diagram of a system architecture for implementing video playing in a multimedia call by a UE1 in an example three according to the second embodiment of the present invention;

FIG. 10 is a diagram illustrating a process of data transmission between a UE1 and a UE2 via an external device according to an embodiment of the present invention;

fig. 11 is a schematic diagram of a terminal interface for a user to select media playing at a terminal or a remote end in an example of the third embodiment of the present invention;

fig. 12 is a schematic diagram of the hardware structure of four user terminals and an access point device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.

Although the current mobile device has a powerful function, it is limited by hardware devices, such as multimedia devices, terminal processor memories, etc., which causes problems of low terminal operation speed, large power consumption, etc., and cannot provide better user experience. The method aims to solve the problems that the bandwidth of the existing mobile equipment is limited, the multimedia capability is limited, the processing capability is limited and the like. The inventor of the application proposes that various operations on the mobile terminal can be realized by using hardware resources of peripheral external equipment, and the performance of the mobile terminal is improved. For example, a video call on the mobile terminal can be realized by using a screen of an external television, a microphone, a high-definition camera and the like, so that great experience is brought to a user. And partial processing tasks on the mobile terminal can be realized by utilizing external equipment, so that the calculation speed of key tasks of the mobile terminal is increased, and the power consumption is reduced.

Example one

As shown in fig. 1, the application environment of this embodiment is that a user terminal UE1 communicates with a base station 1 through a mobile communication network, communicates with an external device AP1 through a short-range wireless communication network, and an AP1 may communicate with devices such as a television through the short-range wireless communication network; accordingly, the user terminal UE2 communicates with the base station 2 through a mobile communication network, communicates with the external device AP2 through a short-range wireless communication network, and the AP2 can communicate with a device such as a television through a short-range wireless communication network. Thus, the UE1 and the UE2 can communicate directly through a mobile communication network, such as 3G, 4G, or 5G, or can communicate through a WLAN and a network, such as the internet, between the AP1 and the AP 2. In this application, a user terminal is also referred to as a mobile terminal or a terminal device. The external devices AP1 and AP2 may be access point devices of a WLAN, or may be other access point devices that can establish a communication connection with a user terminal, a multimedia device, and the like through short-range communication, and communication between different access point devices may be implemented. However, the connection between the external device and the multimedia device is not limited to short-range communication, and the access point device itself may also be the multimedia device, or may also be integrated with the multimedia device, which is not limited in this application.

Based on the above application environment, the present embodiment may implement one or more of the following resource sharing processes:

1. the WLAN and the existing mobile communication network such as a 3GPP network are fused to realize joint transmission, the 3GPP network is in charge of signaling connection, and the WLAN is in charge of data transmission, so that the purposes of load distribution and network performance improvement are achieved.

2. The terminal equipment manages external equipment such as an external large screen, an external microphone, a high-definition camera and the like through the external equipment. External multimedia equipment such as external high-quality microphone speaker, high definition digtal camera are fully utilized to carry out multimedia acquisition and broadcast.

3. The terminal device distributes a part of calculation tasks (such as data transmission, coding and decoding and the like) to the external device to complete, so that the calculation burden is reduced, the power consumption is reduced, and the user experience is improved.

The data transmission method of the present embodiment is shown in fig. 2, and includes the following operations:

s100: after the first user terminal A establishes signaling connection with the second user terminal B, the user terminal A sends the information of external equipment of the terminal to the user terminal B and acquires the information of the external equipment of the user terminal B;

in this embodiment, the signaling connection established between the first user terminal a and the second user terminal B is a signaling connection of a mobile communication service, such as a signaling connection of a video call. The mobile communication service is initiated by the user through the mobile communication network but is not limited to using the mobile communication network.

The external device referred to herein is a device that can provide external hardware resources (such as multimedia resources, storage resources, and logic operation resources) sharing for the user terminal through a short-range communication technology (such as bluetooth and WIFI), for example, the external device is an Access Point (AP) device (also referred to as a hotspot, an AP hotspot, and an AP device) for short-range communication, that is, the first user terminal and the second user terminal are both connected to their respective external devices through short-range communication. Therefore, before the user terminal a establishes the signaling connection with the user terminal B, the information of the external device of the terminal may be acquired through the near field communication technology, and the acquired information of the external device of the terminal includes one or more of the capability description information and the communication identification address. The capability description information of the external device may include media description information of a multimedia device connected to the external device, and the communication identification address of the external device may be an IP address or other information that can be used for communication connection. In one example, the first user terminal acquires information of an external device of the first user terminal through a near field communication technology, and the method includes: the first user terminal sends control signaling for requesting capability description information to an external device of the first user terminal, and the control signaling is indicated by using a reserved field in a data frame of near field communication.

It should be noted that, the manner for the user terminal a to send the information of the external device of the terminal to the user terminal B is at least two as follows:

in the first mode, the user terminal a sends the current control signaling to the user terminal B through the signaling connection, and the reserved field of the control signaling may carry information of the external device of the user terminal a.

The control signaling referred to herein may include any one or more of the following: call request, media modification request, INVITE request, re-INVITE request, UPDATE data UPDATE.

And secondly, the user terminal A sends a data packet such as a WIFI data packet and/or a Real-time transport Protocol (RTP) message to the user terminal B by using a transmission channel established after the transmission of the control signaling is finished, and the sent data packet carries information of external equipment of the user terminal A.

The formats of the WIFI data packet and the RTP packet are not particularly limited. The above transmission method may be used when the user terminal B transmits the information of the terminal external device to the user terminal a.

The user terminal may transmit all information of the external device of the user terminal a in the first or second manner, or may transmit partial information of the external device of the user terminal a in the first manner, and transmit other information of the external device of the user terminal a in the second manner.

S200: the user terminal A indicates the external equipment of the terminal, and establishes data service connection between the external equipment of the user terminal A and the external equipment of the user terminal B according to the exchanged information of the external equipment of the user terminal B;

s300: the user terminal A indicates the external device of the terminal to transmit data through the established data service connection.

In this embodiment, the data transmission may include any one or more of the following data transmissions in a mobile communication service:

sending audio streams collected by multimedia equipment connected with the terminal or external equipment of the terminal;

sending a video stream collected by multimedia equipment connected with the terminal or external equipment of the terminal;

sending files stored in a storage device connected with the terminal or external equipment of the terminal;

receiving the audio stream, and forwarding the audio stream to the terminal or a multimedia device connected with the external device of the terminal for playing;

receiving the video stream, and forwarding the video stream to the terminal or a multimedia device connected with the external device of the terminal for playing;

and receiving the file, and forwarding the file to the terminal or a storage device connected with the external device of the terminal for storage.

In this embodiment, in order to determine which device performs audio and video acquisition and playing, before the first user terminal instructs the external device of the terminal to perform data transmission through the data service connection, the method further includes: the first user terminal determines options to be applied from one or more of the following options according to a user instruction and indicates external equipment of the first user terminal:

playing a video at the terminal, or playing the video at a multimedia device connected with an external device of the terminal;

playing audio at the terminal, or playing audio at a multimedia device connected with an external device of the terminal;

acquiring a video at the terminal, or acquiring a video at a multimedia device connected with an external device of the terminal;

and acquiring audio at the terminal, or acquiring audio at a multimedia device connected with external equipment of the terminal.

The options to be applied may be options selected by the user from the call interface, options selected in a setting interface before the call, and the like.

Before the first user terminal instructs the external device of the terminal to perform data transmission through the data service connection, determining an option to be applied from one or more of the following options according to a user instruction and instructing the external device of the terminal:

When the data transmission does not involve the multimedia equipment connected with the external equipment, the user terminal uses the communication resource of the AP equipment to complete the data transmission in the mobile communication service, and when the data transmission involves the multimedia equipment connected with the external equipment, the user terminal also uses the multimedia resource of the multimedia equipment to realize the collection and playing of multimedia.

In one example, the indication that the user terminal a sends to the external device of the terminal may be an indication that the external device of the terminal starts sending voice and/or video media streams (i.e. audio streams and/or video streams) in mobile communication to the second user terminal through the data service connection; and instructing the external equipment of the terminal to start receiving the voice and/or video media stream sent by the second user terminal through the external equipment of the second user terminal through the data service connection. In another example, the external device of the terminal may also be simultaneously instructed to forward the received voice and/or video media stream to the terminal for playing, or to forward to the multimedia device connected to the external device of the terminal for playing.

The data transmission method of the embodiment can be described simply as follows: 1. mobile terminals A (namely user terminals) and B finish information exchange with external equipment A and B respectively through a near field communication technology, and information of the external equipment A and B is acquired, wherein the information comprises capability description information, a communication identification address and the like; 2. the mobile terminal A and the mobile terminal B establish signaling connection and transmit relevant information of respective external equipment; 3. the external devices a and B establish data connection to complete data transmission, and optionally, the external devices are responsible for voice and video acquisition and playing through the connected multimedia devices.

The control information transmission formats, i.e. the control signaling formats, referred to in the above-described method are described below, and these control information transmission formats may also be used in other embodiments. The method and the device need to transmit the self-defined control signaling, and carry information of external equipment (such as AP equipment) through the control signaling, wherein the information comprises media description information, IP addresses and the like. The control signaling in this context includes at least control signaling between the UE based on base station communication, and may also include control signaling between the UE based on short-range communication and an external device. The transmission of the control signaling may adopt two schemes as follows:

1. the self-defined control signaling is transmitted through the control signaling defined in the protocol, namely, the reserved field in the existing control signaling is utilized, and the control field is written in the reserved field to describe the media description information and the IP address.

2. And transmitting through a transmission channel established after the control signaling is finished, packaging the control signaling into a corresponding data packet by using a data packet such as a wifi data packet and an RTP message (namely an RTP data packet), and transmitting, wherein the opposite end needs to decode.

The transmission of UE and UE control information by base station communication is illustrated as follows:

the Voice over LTE is an IP data transmission technology implemented based on an IP Multimedia Subsystem (IMS). VoLTE does not need a 2G/3G network, and all services are borne on a 4G network.

In the process of establishing a call, when a session initiator initiates a session request, control information (i.e. information of external equipment of the UE) is carried in a media line of a voice media; the media capability negotiation is performed by adding a special media line carrying control information in a Session Description Protocol (SDP) to help each external device establish data connection. The SDP description may consist of a number of lines of text, as in table 1 below, the underlined fields may each be used to transmit customized control information, which in this example includes the IP address of the external device, but may also include capability description information.

Table 1 is a table of meaning of SDP fields

The SDP format is illustrated here as follows:

v＝0

o＝originator 2890844526 2890842808IN IP4controller.example.com

s＝-

c＝IN IP4controller.example.com

m＝application 10.12.50.22 49153TCP

a＝setup:active

a＝connection:new

m＝audio 49170RTP/AVP 0

in table 1, it can be seen that the underlined field, m ═ application 10.12.50.2249153 TCP, is a media channel that can be used to indicate corresponding multimedia information, and includes the IP address and port number of the external device.

Based on table 1, a process of transmitting control signaling between UE1 and UE2 is illustrated, as shown in fig. 3, which includes a Session Border Controller (SBC) and an IP Multimedia Subsystem (IMS), and the process includes:

the UE1 sends an INVITE call request, denoted as "INVITE SDP Offer" (Offer) to the UE2, which carries a media line (e.g., line m) that may carry voice media for a call in the Session Description Protocol (SDP).

SBC1 forwards INVITE SDP Offer to IMS.

The IMS forwards INVITE SDP Offer to the SBC2 on the called subscriber side.

SBC2 forwards INVITE SDP Offer to UE 2.

UE2 processes the received INVITE SDP Offer, constructs an SDP, and sends the SDP to SBC2, where the constructed SDP may also include the IP address of the external device of UE 2; from the media negotiation perspective, the SDP is Answer, i.e., the UE2 returns a 200OK SDP Answer.

SBC2 forwards a 200OK SDP answer message to the IMS.

IMS forwards the 200OK SDP answer message to SBC 1.

SBC1 forwards a 200OK SDP answer message to UE 1. At this point, the media channel for the voice call is established.

The UE1 exchanges control information (protocol-specified or customized control information) and negotiates capabilities (e.g., multimedia capabilities) with the UE2 via SDP within a media channel.

Because the established media channel is transmitted based on Real-time Transport Protocol (RTP), the current network side network element supports the function, and no additional requirement of the network side exists, so that the media channel is convenient to be rapidly deployed and popularized in the network. And the mode is convenient for standardization and industrialization.

In the second mode, after the formal set-up of the VoLTE call is completed, the UE1 and the UE2 start the VoLTE call, both parties use the RTP protocol to transmit data packets, and can encapsulate the control signaling into RTP packets to transmit to the opposite end, and the opposite end receives the data packets and then decodes the data packets. The control signaling carries information of the external device, such as communication identification address and/or capability description information. The control signaling may include control signaling between the base station communication-based UE and control signaling between the near field communication-based UE and the external device.

The transmission of control signaling between the UE and the external device through the near field communication is described as follows:

the 802.11 frame is divided into three parts (as shown in fig. 4): frame header (Mac header), Frame entity (body), Frame Check Sequence (FCS). All frames start with a Frame Control field that is two bytes long, and may further include a Duration identifier (Duration ID), a target Address (Dst Address), a source Address (Src Address), and the like, where the Frame Control field includes the following subfields: protocol (Protocol), Type (Type), Subtype (Subtype). The Subtype values 0110-0111 and 1101-1111 of the management frame are not used at present. This field may be used to add control information as shown in table 2.

Table 2 shows control signaling identification table represented by fields 1101-1111 of management frame

The user terminal may set a value of Subtype of the management frame to 1101 and transmit to the AP device to indicate that capability description information is requested from the AP device.

Several examples of the present embodiment are described below

Example 1

The user 1 and the user 2 want to carry out video call, the user can carry out video call after returning home, the video can be played by using a television screen, a professional camera in a living room is used for shooting, and a video picture on the television can be switched back to the terminal device (namely a mobile phone) for display after the television is closed.

The system architecture for implementing multimedia call between UE1 and UE2 according to the present exemplary method is shown in fig. 5, and the whole process includes the following operation steps:

step one, a user selects to initiate a request for using external equipment, and the UE1 requests the AP1 equipment for capability description information, communication addresses and the like of the external equipment according to a user instruction;

the multimedia capability of the multimedia device connected with the AP1 is regarded as the capability of the AP1, and the capability description information of the AP1 includes the media description information of the multimedia device connected with the AP 1.

And step two, the AP1 equipment acquires the media description information of all the connected multimedia equipment, including display camera parameters, screen parameters, microphone parameters, loudspeaker parameters and the like. If the current multimedia device is online, i.e. in an open and connected state, the AP1 device returns media description information of all online multimedia devices to the UE1 device; the AP1 equipment acquires the communication address of the AP1 equipment in the network and returns the communication address to the UE1 equipment, such as an IP address;

step three, the UE1 initiates a communication request with the remote device UE2 to establish a control signaling connection, such as dialing a phone call. The information of the AP1 device, including media description information and IP address, is carried by the control signaling. The control signaling may include one or more of: a call request, a media capability, an INVITE request, capability UPDATE data UPDATE, an IP address carrying an external device, an IP address of a home terminal UE, and the like;

the process of initiating a communication request to the UE2 by the UE1 and establishing a control signaling connection is basically the same as that in the prior art, but the difference is mainly that information of the AP1 device needs to be carried in the control signaling sent by the UE 1.

After receiving the communication request, the UE2 requests information from the AP2, where the information of the AP1, such as an IP address, may be carried in the request; after obtaining the information (such as the IP address and the capability description information) of the AP2 returned by the AP2, the UE2 carries the information of the AP2 in the control signaling response returned to the UE 1.

Step four, after receiving the control signaling response of the UE2, the UE1 obtains the information of the AP2 of the external device connected to the UE2, and instructs the UE1 to establish a data service connection between the AP1 device connected through close range communication and the AP2 device connected through close range communication by the UE 2. The voice and/or video media stream sent by the UE1 to the UE2 is transmitted to an external device of the AP2 device by the AP1 device, and the voice and/or video media stream of the UE2 to be received by the UE1 is sent to the AP1 device by the AP2 to be received and is distributed to the multimedia device by the AP1 device to be played.

When the UE1 instructs the AP1 device and the AP2 device to establish a data service connection, information of the AP2, such as a communication identification address of the AP2, media description information of a multimedia device connected to the AP2, and the like, may be sent to the AP1 device. The indication may also be regarded as an indication for transmitting data, and the AP1 device may start transmitting data by establishing a data service connection with the AP2 device. The UE1 may send an indication to start data transmission after instructing the AP1 device and the AP2 device to establish a data traffic connection, the AP1 device receives the indication to start data transmission again,

the method of the embodiment fuses a near-distance communication network such as WIFI and Bluetooth with a mobile communication network such as a 3G, 4G and 5G network to realize joint transmission, the mobile communication network is in charge of signaling connection, and an access point device of the near-distance communication network is in charge of data transmission, so that the purposes of load distribution and network performance improvement can be achieved, such as improvement of transmission bandwidth and reduction of communication cost, and user experience can be improved for occasions with limited mobile communication network bandwidth. In addition, the user terminal manages external devices such as an external large screen, an external microphone and a high-definition camera through the external devices. External multimedia equipment such as external high-quality microphone speaker, high definition digtal camera can make full use of carry out multimedia acquisition and broadcast, improve user experience. For example, if a user can make a video call on a television, external multimedia devices such as an external high-quality microphone and a high-definition camera are fully utilized, which brings great experience progress to the user.

It should be noted that, the intelligent splitting is performed during the call, three user experiences, that is, three connection options, are provided, fig. 6 shows a selection interface related to the user terminal, where the three options are:

1. when the option is selected, the UE1 initiates a communication request with the remote device UE2, and the offloading process is automatically performed.

2. The suggestion is connected, when satisfying intelligent reposition of redundant personnel condition promptly, and the suggestion guides the reposition of redundant personnel. When the option is selected, after the UE1 initiates a communication request with the remote device UE2, it is provided on the terminal interface whether the user performs offloading, that is, data transmission is performed through the AP, and after the user determines that the offloading is performed, the offloading process is performed.

3. And manual connection, namely setting no active shunting in the notification bar. When the option is selected, the UE1 initiates a communication request with the remote device UE2, and then performs no offloading processing or prompting unless the user selects to perform offloading via the UE interface.

Example two

The method for transmitting the files between the user 1 and the user 2 by the computer comprises the following operation steps:

step one, a user 1 selects to initiate a request for using an external device, and a UE1 requests information of an AP1 device, such as capability description information, a communication identification address and the like, from an AP1 device;

step two, the AP1 device obtains media description information of all connected multimedia devices, where the multimedia devices include a computer a, and the AP1 device returns the media description information of the computer to the UE1 device; the AP1 equipment acquires the communication identification address of the AP1 equipment in the network and returns the communication identification address to the UE1, such as an IP address;

step three, the UE1 initiates a communication request with the remote device UE2, e.g., to place a call, to establish a control signaling connection. The control signaling sent to the UE2 carries the information of the AP1 device, including the media description information of the computer a and the IP address of the AP1, etc.;

step four, after receiving the control signaling response of the UE2, the UE1 obtains the information of the AP2 of the external device connected to the UE2, instructs the UE1 to establish a data service connection between the AP1 device connected through near field communication and the AP2 device connected through near field communication by the UE2, transmits the file at the designated position on the computer a to the computer B connected to the AP2 device, and stores the file in the computer B.

In this step, the files and locations to be transmitted may be selected by the UE1 and indicated to the AP1, and the files may be saved in the default location or designated location of the computer B.

The computer in this example is used as a storage device for files, and thus can be regarded as a storage device, that is, this example realizes file transfer between a storage device to which the AP1 device is connected and another storage device to which the AP2 device is connected. It is readily understood that files may also be transferred from the UE1 to a storage device to which the UE2 or AP2 is connected.

Example two

The embodiment provides a resource sharing method.

Fig. 7 shows a resource sharing method at a ue side in this embodiment, which includes:

step S400, the user terminal requests the information of the external device to the external device through near field communication;

in this embodiment, the external device may be an access point device of a short-range communication network, but is not limited thereto.

Step S500, the user terminal receives the information of the external equipment returned by the external equipment, and uses peripheral external hardware resources according to the information of the external equipment.

In this embodiment, the information returned by the external device includes one or more of capability description information and a communication identification address of the external device; wherein the capability description information of the external device may include one or more of media description information of a multimedia device to which the external device is connected and arithmetic capability information of a logical arithmetic device to which the external device is connected. The communication identification address of the external device may be an IP address or the like.

In an example of this embodiment, the user terminal uses peripheral external hardware resources according to the information of the external device, and may adopt the manner of the first embodiment, that is, data transmission in a mobile communication service is performed between the external device of the user terminal and the external device of a remote user terminal.

In another example of this embodiment, the using, by the user terminal, peripheral external hardware resources according to the information of the external device includes:

the user terminal accesses the network server through external equipment;

and the user terminal instructs the external equipment to forward the media stream received from the network server to the multimedia equipment connected with the external equipment for playing.

By the example, after the user terminal establishes connection with the server on the network side through the mobile communication network, the video resources issued by the server can be played by using peripheral devices such as a large display screen, a high-quality sound box and the like.

the user terminal establishes signaling connection with a server through a mobile communication network, and sends the communication identification address of the external equipment to the server through the signaling connection, so that the server establishes data service connection with the external equipment according to the communication identification address of the external equipment and receives a media stream collected by multimedia equipment connected with the external equipment.

By the example, after the user terminal establishes a connection with the server of the data platform, for example, a call is initiated, the server may receive the audio/video media stream collected by the multimedia device around the user terminal.

In the above example, the user terminal sends the information of the external device to the server through the signaling connection, and one or more of the following manners are adopted:

in a first mode, the user terminal establishes signaling connection with the server through a mobile communication network, sends a control signaling to the server through the signaling connection, and carries information of the external equipment in a reserved field of the control signaling;

in a second mode, the user terminal establishes signaling connection with the server through a mobile communication network, and sends a data packet to the remote device by using a transmission channel established after the transmission of the control signaling is finished, wherein the data packet carries information of the external device.

The user terminal may transmit all information of the external device in the first or second manner, or may transmit partial information of the external device in the second manner, and transmit other information of the external device in the second manner.

the user terminal distributes a calculation task to the external equipment so as to execute the calculation task through a logic operation device connected with the external equipment;

and the user terminal receives a calculation result returned by the external equipment.

By the example, the user terminal can distribute a part of computing tasks (such as data transmission, encoding and decoding and the like) to peripheral equipment to complete, so that the computing burden of the user terminal is reduced, the computing speed of key tasks of the terminal is increased, the power consumption is reduced, and the user experience is improved.

As shown in fig. 8, the resource sharing method on the external device side in this embodiment includes:

step S600, the external device receives an information request sent by the user terminal through near field communication;

in this embodiment, the information request is in the form of a data frame, and the data frame carries a beacon requesting a control signaling of capability description information.

Step S700, the external device acquires resource information of a peripheral device connected to the external device, and returns the resource information of the peripheral device to the user terminal as capability description information of the external device, so that the user terminal uses the resource of the peripheral device.

In this embodiment, the external device is an access point device of a near field communication network, but the application is not limited thereto; the resource information of the peripheral equipment comprises one or more of media description information of the multimedia equipment and arithmetic capability information of the logic arithmetic equipment.

In this embodiment, before or while the external device returns the capability description information to the user terminal, the external device may also return a communication identifier address of the external device to the user terminal. The capability description information and the communication identification address may be returned to the user terminal in response to the same request or may be returned to the user terminal in response to different requests.

In an example of this embodiment, after the external device returns the capability description information of the peripheral device to the user terminal, the method further includes:

and the external equipment establishes data service connection of remote equipment communicated with the user terminal according to the indication of the user terminal and carries out data transmission through the data service connection.

The remote device may be a remote external device, and the remote external device is connected with the user terminal through a remote terminal called by mobile communication through near field communication; at this time, the external device performs data transmission through the data service connection, including:

the external equipment and the remote external equipment perform point-to-point data transmission based on a peer-to-peer protocol networking real-time media stream protocol (RTMFP); or

The external equipment and the remote external equipment realize data transmission between each other through the server based on a real-time messaging protocol (RTMP); or

And the data transmission between the external equipment and the remote external equipment is realized based on a real-time transport protocol (RTP) and a real-time transport control protocol (RTCP).

However, the remote device may also be another user terminal, or a server on the network side, or other devices. The user terminal may load the data transmission with the remote device to be performed by an external device, for example, the external device may perform transmission of an audio stream and/or a video stream of a video call. Thereby achieving the effects of load sharing, bandwidth improvement, network transmission performance improvement and the like.

In this example, the collection and playing of the audio stream and/or the video stream may also be implemented by an external device, and also by a multimedia device connected to the external device.

The data transmission comprises any one or more of the following: sending an audio stream collected by the user terminal or the multimedia device connected with the external device; sending a video stream collected by the user terminal or the multimedia equipment connected with the external equipment; sending a file stored in the user terminal or the storage device connected with the external device; receiving audio stream, and forwarding the audio stream to a user terminal or multimedia equipment connected with the external equipment for playing; receiving the video stream, and forwarding the video stream to a user terminal or multimedia equipment connected with the external equipment for playing; and receiving the file, and forwarding the file to a user terminal or a storage device connected with the external device for storage.

The user terminal may specify which device (e.g., the user terminal or the multimedia device) transmits the audio stream and/or the video stream collected by the external device while transmitting the indication for data transmission, and may also specify which device (e.g., the user terminal or the multimedia device) forwards the received audio stream and/or the received video stream.

In this example, before the external device establishes the data service connection with the remote device communicating with the user terminal according to the indication of the user terminal, the method may further include: the external equipment receives an encryption key sent by the user terminal through near field communication, wherein the encryption key is generated in the process of authentication with a network side after the user terminal initiates a call request; and when the external equipment carries out data transmission through the data service connection, the encryption key is adopted to encrypt the data to be transmitted.

In another example of this embodiment, after the external device returns the capability description information of the peripheral device to the user terminal, the method further includes:

the external equipment receives an instruction for executing a calculation task sent by the user terminal, and forwards the calculation task to the logic operation equipment connected with the external equipment for execution;

and the external equipment receives the calculation result returned by the logic operation equipment and returns the calculation result to the user terminal.

In this example, the external device may accept a computing task of the user equipment, increase an operation speed of the user equipment, and save power for the user equipment.

Several examples of the present embodiment are explained below.

Example three

In this example, the UE1 may implement video playing in a multimedia call through a multimedia device connected to an AP device, and the system is shown in fig. 9, where in an example, a flow of a user playing a network video is as follows:

step one, a user selects and initiates a request for using external equipment through UE;

step two, the UE requests capability description information, a communication identification address and the like to an external device AP;

step three, the external equipment AP acquires the media description information of all connected multimedia equipment and the IP address of the AP in the network and returns the media description information and the IP address to the UE equipment;

step four, the user selects to open a streaming media content link on the UE interface, clicks a media address in a webpage and sends a playing request;

step five, the UE sends a get request message to the AP;

step six, the AP sends a get request message to a Media Server (Media Server);

step seven, after receiving the get request message, the Media Server sends a 200OK response message containing the SDP file;

step eight, the AP receives the 200OK response message, obtains the description information SDP file of the media content and sends the description information SDP file to the UE; the UE analyzes the SDP file, wherein the SDP file comprises information such as media content format information, playing time (npt, such as npt, which is 0-127.8667 unit in second) and the like;

step nine, the AP sends a SETUP request message according to streamID or trackID in the SDP, and a transmission mechanism of the video stream is determined firstly;

step ten, after receiving the SETUP request message of the video stream, the Media Server sends a 200OK response message, which comprises a session identifier and a transmission protocol (RTP) port number of the player;

step eleven, the AP receives the 200OK response message, sends a SETUP request (SETUP request) message according to the stream identifier (streamID) or the tracking identifier (trackID) in the SDP, and then determines the transmission mechanism of the audio stream;

step twelve, after receiving the SETUP request message of the audio stream, the Media Server sends a 200OK response message, including a session identifier, a transmission protocol (RTP) port number of the player, and the like;

step thirteen, the AP receives the 200OK response message and sends a PLAY request message, wherein the address of the media to be played is contained in the PLAY request message.

Fourteen, Media Server receives 200OK response message, transmits Media content to player through RTP, and transmits Real-time Transport Control protocol (RTCP) packet;

step fifteen, the AP receives RTP and RTCP packets. The system displays the buffer, decodes and starts to play the streaming media content. Sending RTCP data packet to Media server in playing process;

sixthly, the AP distributes the received voice and/or video media streams to a plurality of connected multimedia devices for playing.

Before this step, the user may select to play the voice and/or video media stream on the AP-connected multimedia device, such as a television, and instruct the AP through the UE interface, which may be selected after sending a play request by clicking a media address in a web page, or may be selected before this step, for example, the UE may provide a selection to play a video on the UE or play a video on a peripheral multimedia device for the user to select.

The user terminal can manage external devices such as an external large screen, an external microphone, a high-definition camera and the like through the external devices. The external multimedia equipment such as external high-quality microphone speaker, high definition digtal camera can be fully utilized to play multimedia, and user experience is improved.

Example four

In this example, when a user suddenly encounters an emergency and needs to dial a 911 call, a 911 acceptance center obtains a video picture around the location of the user after the 911 call is dialed, and the process includes the following steps:

step 1, a user selects to initiate a request for using external equipment, and UE automatically requests capability description information, communication addresses and the like from surrounding AP equipment;

step 2, the AP equipment acquires the media description information of all connected multimedia equipment, wherein the multimedia equipment comprises a public security camera on the road, and the AP equipment returns the media description information of the public security camera to the UE equipment; the AP equipment acquires a communication identification address (such as an IP address) of the AP equipment in a network and returns the communication identification address to the UE;

the security camera of this example has the ability to interact with the AP device.

And step 3, the UE emergently calls 911, initiates a communication request with a 911 call center, and establishes control signaling connection. Carrying information of the AP equipment, such as media description information, IP address and the like of a camera, through the control signaling;

and 4, after receiving the control signaling of the UE, the 911 call center acquires the information of the AP of the external equipment connected with the UE, directly establishes data service connection between the 911 call center and the AP equipment connected with the UE, or indicates the AP equipment connected with the 911 call center and the AP equipment connected with the UE to establish data service connection, and receives a real-time video stream acquired by the security camera and sent by the AP equipment connected with the UE. The 911 call center can see real-time video information around the user.

The processing performed by the 911 call center in the above steps is implemented by an associated server of the 911 call center.

By the example, the call center can directly acquire the real-time video stream from the peripheral security cameras under the condition that the user terminal cannot send the real-time video stream due to limited data flow, which is equivalent to the hardware resources of the peripheral security cameras used by the user terminal. The shooting angle of the security camera is different from that of the user terminal, so that the 911 call center can master the field situation more comprehensively.

EXAMPLE III

The embodiment further takes an LTE network as an example to describe a specific implementation of the method for implementing data transmission between the two user terminals through respective APs. The implementation mechanism in the LTE network environment is the same as that in the 3G and 2G networks.

In the LTE network, a process of implementing data transmission between the UE1 and the UE2 through an external device is shown in fig. 10, and includes the following operations:

step one, the UE1 completes IMS network registration, specifically, executes an attach process after power on and network selection, establishes an IMS network PDN connection and a default bearer after registration is completed, and may obtain the following addresses in the registration process:

IP address of U1E: the address is obtained by network allocation in PDP context activation and is used for determining the self address during registration.

IP address of network server: the address is obtained by network allocation in PDP context activation and is used for discovering the address of the server in the registration process.

The UE1 may obtain the network server address in several ways:

1) in a signaling mode, in the LTE registration process, the UE requests the address of the CSCF in the PDN Connectivity Request, and the network informs the address of the CSCF of the UE by activating a cell in a bearer.

2) DHCP mode: generally applied to a conventional wired network, an IP address of a server assigned to the UE in a DHCP manner.

3) Providing, by the UE: this way, the UE uses the configuration file or variable associated with the network server when initiating the service by presetting the configuration file or variable into the UE or the UICC card.

Requesting capability description information from the external device before the UE1 device communicates with the remote device; the external device is a device with short-range communication capability, in this example, an AP1 hotspot device.

The UE1 requesting information from the AP1 device, e.g., sending a capability query request message;

the AP1 device obtains the media description information of all connected multimedia devices (e.g., camera, microphone, screen, speaker, etc.), including display camera parameters, screen parameters, microphone parameters, speaker parameters, etc. If the current multimedia device is online, i.e. in an open and connected state, the AP1 device returns media description information of all online multimedia devices to the UE1 device; the AP1 equipment acquires a communication identification address of the AP1 equipment in the network, such as an IP address, and returns the communication identification address to the UE 1; the AP1 may return the media description information and IP address to the UE1 via a capability description response.

And step three, the UE initiates a VoLTE calling and triggers the establishment of the initial session.

In this step, the network server sends a call request to the UE2 after receiving the call request sent by the UE1, and sends a call response to the UE1 according to the call response of the UE 2. In this process, the communication between the UE1 and the network server may include the following operations:

1) the calling terminal (i.e., UE1) sends an INVITE request;

2) the network server sends a response message to the terminal (i.e., UE1) indicating that the network has received the INVITE request of the terminal;

3) the network server sends RINGING 180 message to the terminal (i.e. UE1) indicating that the opposite terminal starts RINGING;

4) the network server replies OK 200 to represent that the opposite terminal receives the call and enters the conversation state

Fourthly, capability exchange and control plane signaling interaction are carried out between the calling terminal and the called terminal;

in this step, UE1 and UE2 perform media capability negotiation. After the UE1 initiates a control signaling connection with the remote device UE2, the information of the AP1 device, including media description information and IP address, may be carried by the control signaling. The control signaling comprises one or more of the following: the call request, the media capability, the INVITE request, the capability UPDATE data UPDATE, and the control signaling carry the IP address of the external device, the IP address of the home terminal UE, etc.

After receiving the control signaling response of the UE2, the UE1 obtains the information of the AP2, which is the external device connected to the UE2, including the IP address of the AP 2.

And step five, finishing the session establishment and starting data transmission.

The UE1 establishes a data traffic connection between the AP1 device and the AP2 device. The voice and/or video media stream sent by the UE1 to the UE2 is transmitted to an external device of the AP2 device by the AP1 device, and the voice and/or video media stream AP2 of the UE2 to be received by the UE1 is sent to the AP1 device for receiving and is distributed to a multimedia device by the AP1 device for playing.

In this embodiment, the AP device splits received information, such as a voice and/or video media stream, into data packets according to the media description information of the multimedia device; transmitting each data packet to each multimedia device; and splitting information to be sent, such as voice and/or video media streams, into a plurality of data packets according to the determined media description type to which each data packet belongs, and numbering the data packets, for example, the AP device integrates the data packets belonging to the same information, and sends the integrated data to the target device, so that each data packet carries its own number.

In this embodiment, after media capability negotiation between UEs, the updated media description information may also be sent to a multimedia capability update signaling to a server according to a change of the media description information fed back by the AP device; and after receiving the response of the multimedia capability updating signaling, the UE interacts with the remote equipment according to the updated multimedia function. The multimedia capability update signaling comprises one or more of the following: media modification request, re-INVITE request, UPDATE data UPDATE.

The following describes data transmission between AP devices in this embodiment, and the method for data transmission between AP devices in this embodiment can also be applied to the corresponding scenarios in the foregoing embodiments.

In an example of this embodiment, the AP devices communicate with each other based on a video P2P of an RTMFP Protocol, the RTMFP Protocol is all called as a peer-to-peer Protocol networking Real-Time Media Flow Protocol (Real-Time Media Flow Protocol), the Protocol is based on udp (user Datagram Protocol), and allows the hotspots (i.e., the AP devices) to establish peer-to-peer communication connection therebetween, and the users may directly establish connection therebetween without a server, thereby implementing data transmission between hotspots. The basic working principle and the implementation process are described as follows: before direct data transmission, the hotspot must be connected to a Server supporting an RTMFP protocol to obtain a unique ID as a unique identifier of the hotspot, and the identifier can also be used as a communication identification address of the hotspot; the hotspot may then distribute audio, video or data information via UDP streams. Any other hot spot connected to the same server can directly receive various kinds of stream information issued by the previous hot spot according to the ID. Each client connected to the server is treated as a node (peer), and a plurality of nodes visible to each other form a group (group) at the server. The group simultaneously stores the routing information among the members, so that a direct or indirect connection path exists between any two nodes in the group.

The hotspot can use the user name of the hotspot to connect with the server, and after the connection is successful, a unique Peer ID is obtained; a successfully connected user can make a communication request for the corresponding user according to the registration name of another user; the user receiving the request may choose to accept or decline the invitation; after the connection is established, the two users can send text information to each other besides audio and video calls. After the hotspot successfully connects the server, inputting a user name to be contacted, sending a Call instruction, inquiring a corresponding user ID from the hotspot ID management service according to the invited user name by the server, and if the user ID does not exist, failing to invite; if the user ID is successfully inquired, connection establishment with the corresponding user ID is attempted. If the other side refuses the invitation, the communication is failed to be established; if the opposite side accepts the invitation, communication is established, and the sending and receiving of the stream are started. After the user chooses to end the call, the user name is logged off from the hotspot ID management service first, and then the streams for sending and receiving are cut off respectively.

The RTMFP protocol allows direct communication between hotspots, reducing communication delays, and is therefore preferred.

In another example of the embodiment, all hotspots transmit data through the server based on the RTMP protocol between the AP devices. The RTMP Protocol is known as Real time messaging Protocol (Real time messaging Protocol), a TCP/IP Protocol designed specifically for transmitting video, audio and data, and can maintain a persistent connection with a server at a hotspot and allow Real-time communication. The protocol defines a plurality of channels that can act independently, each responsible for a different function. According to the connection environment of the hotspot and the service end, the RTMP protocol divides data such as audio and video into data packets with certain size, adds header information into the data packets, specifies a channel ID to which the data is transmitted, and if necessary, notes a timestamp generated by the data, and then transmits the data.

In yet another example of this embodiment, the data transfer between the AP1 device and the AP2 device may be built on top of the TCP protocol or UDP protocol, enabling bi-directional RTP/RTCP streaming, one for audio data transfer and one for video data transfer. RTP is the transport protocol for media payloads. RTP provides time information and stream synchronization for end-to-end real-time transport over the Internet, but does not guarantee quality of service, which is provided by RTCP. The RTP data protocol is responsible for packetizing streaming media data and implementing real-time transmission of media streams.

During the RTP session, each participant periodically transmits RTCP packets. RTCP packets contain statistics of the number of packets sent, the number of packets lost, etc., so that each participant can use this information to dynamically change the transmission rate, and even the payload type. RTP and RTCP are used in conjunction, which can optimize transmission efficiency with efficient feedback and minimal overhead.

Session Description Protocol (SDP) offers/replies to video media are formatted as specified in 3GPP TS 26.114, section 6.2.3.

When the application establishes an RTP session, the application will determine a destination transport address. The destination transport address consists of a network address and a pair of ports, and has two ports: one for RTP packets and one for RTCP packets so that RTP/RTCP data can be correctly transmitted.

The RTP transmission process is as follows, and the reception process is reversed.

1) The RTP protocol receives a media information code stream (such as H.263) from an upper layer and encapsulates the media information code stream into RTP data packets; the RTCP receives control information from an upper layer and encapsulates it into an RTCP control packet.

2) RTP sends RTP data packet to even port in UDP port pair; RTCP sends RTCP control packets to odd ports in the UDP port pair.

The synchronization of the sound and the picture can be achieved based on the relative time of the sound stream and the picture stream (i.e., the time stamp of the RTP packet), and their absolute time (i.e., the RTCP in the corresponding RTCP packet).

Next, a security policy for data transmission between AP devices according to this embodiment is described. The security policy of the present embodiment can also be applied to the corresponding scenarios of the foregoing embodiments.

Since the UE uses the authentication information during the registration process and all subsequent signaling is protected. Therefore, when the media stream is transmitted from the AP1 device to the AP2 device, the UE may transmit the authentication information to the AP1 device through the wifi link, and the AP1 device may use the authentication information to increase security of data transmission between the AP and the AP.

In one example, after the UE initiates a call request, the UE performs authentication with the network side, and the network side selects an authentication vector group and sends RAND and AUTN therein to the UE. And after receiving the RAND (absolute) AUTN, the UE performs authentication processing on the USIM side and verifies the AUTN. In the step, after the authentication of the user to the network is performed, the USIM calculates XMAC and compares it with the MAC value in AUTN, so that the user completes the authentication of the network. The USIM calculates RES (user response), sends to the network side, compares whether RES is equal to XRES, if equal, the network authenticates the user identity, and the user at the network side completes the authentication of the user, which is called bidirectional authentication.

The key generated in the above-mentioned two-way authentication process, such as the ciphering key CK, is shared between the core network and the UE. The UE can send the CK to the connected AP equipment through near field communication such as WIFI (wireless fidelity), the CK is transmitted in a security mode of near field messages, and the data information to be transmitted can be encrypted by using the CK parameters through communication between the AP and the remote AP after the AP obtains the CK. For example, the encryption mechanism is to generate a key stream (pseudo-random mask data) by using an encryption algorithm f8, perform bit-by-bit addition on plaintext data and the mask data to generate a ciphertext, transmit user data on a wireless link directly connected to the AP in the form of the ciphertext, and after receiving the ciphertext, the AP on the receiving side performs bit-by-bit addition on the ciphertext and the mask data (the same as the input parameter during encryption, and thus the generated mask data is the same) to restore the ciphertext and the mask data to plaintext data, that is, to decrypt the plaintext data.

The following describes in detail the interaction process in the video call between the user, the UE and the external device, and the user can select whether to use the external device for data transmission and video playing.

In one example, the video call flow under normal conditions is as follows:

step one, user 1 selects to initiate a request for using external equipment through UE 1;

step two, the UE1 requests capability description information, communication address, etc. to the external device AP 1;

step three, the external device AP1 obtains the media description information of all connected multimedia devices and the communication address of the AP1 in the network and returns the media description information and the communication address to the UE1 device;

step four, the user 1 selects to send a call establishment request to the UE2 through the UE 1;

step five, after receiving the call request message, the UE2 sends a response message;

step six, the UE1 receives the response message forwarded by the server, the UE1 and the UE2 control the signaling establishment to be completed, and the interface displays a call interface;

step seven, the user 1 selects and uses external equipment (an external screen, an external camera and the like) on the call interface;

and step eight, the UE1 carries the information of the AP1 equipment through control signaling, wherein the information comprises media description information, an IP address and the like. To the UE 2;

step nine, after receiving the control signaling of the UE1, the UE2 obtains information of the AP2 of the external device connected to the UE2, including capability description information, an IP address, and the like, and sends the information to the UE 1;

step ten, the AP1 device sends an SDP session request message to the AP2, for example, an SDP request message, which contains media capability description of the AP1 device, i.e., an SDP file;

prior to this step, the UE1 may send information of the external device AP2 sent by the UE2 to the AP1 device, instructing the AP1 to start data transmission.

Step eleven, after receiving the SDP request message, the AP2 device sends a 200OK response message, which contains the media capability description of the AP2 device, i.e., an SDP file;

step twelve, the AP1 receives the response message of the AP2 and analyzes the SDP file;

step thirteen, the AP1 sends a SETUP request message according to the stream identifier (streamID) or the tracking identifier (trackID) in the SDP, and determines the transmission mechanism of the video stream firstly;

fourteen, after receiving the SETUP request message of the video stream, the AP2 sends a 200OK response message, including a session identifier, a transport protocol (RTP) port number of the player, and the like;

fifteenth, the AP1 receives the 200OK response message, sends a SETUP request message according to streamID or trackID in the SDP, and then determines the transport mechanism of the audio stream;

sixthly, after receiving the SETUP request message of the audio stream, the AP2 sends a 200OK response message, which includes a session identifier, a transport protocol (RTP) port number of the player, and the like;

seventhly, the AP1 receives the 200OK response message, transmits media content to the player through RTP, and sends an RTCP packet;

eighteen, the AP2 receives the 200OK response message, transmits the media content to the player through RTP, and sends an RTCP packet;

nineteenth, the AP1 device receives RTP and RTCP packets. The system displays the buffer, decodes and starts playing the media content. RTCP packets are also sent to the AP2 during playback;

twenty, the UE1 displays the call interface to play audio and video by using external equipment such as a television;

the external equipment is used for playing the audio and video to serve as a default option, and a user can also select the external equipment to play the audio and video through the call interface. Fig. 11 shows a user interface for a call. The user call interface lists a plurality of options for the user to select media playing at the terminal or the far end. If the terminal screen (playing video) can be selected, a remote screen (playing video) can also be selected; a terminal microphone (for collecting audio) can be selected, and a remote microphone (for collecting audio) can also be selected; terminal shooting (video acquisition) or remote shooting (video acquisition) can be selected; terminal audio (playing audio) can be selected, and remote audio (playing audio) can also be selected. The term "far-end screen", "far-end microphone", "far-end camera", and "far-end sound" as used herein refer to multimedia devices around the terminal, not devices on the opposite side of the network.

In step twenty-one, the AP1 device distributes the received voice and/or video media stream to a plurality of connected multimedia devices, such as televisions, for playback.

In the process of video call, a user can switch and select the resources of the terminal to be used for media stream processing, or select the resources at the far end to be used for media stream processing. In one example, a user switches a video from a television to a mobile phone screen for display (i.e., playing) through a terminal interface, and the following steps are adopted:

step one, a user 1 selects a screen of a using terminal to play, and switches from a television to a mobile phone screen;

step two, the UE1 sends a video switch request message to the AP 1;

step three, the AP1 receives the control information transmitted by the UE1 and sends the control information to the AP 2;

step four, after receiving the video switch request message, the AP2 sends a 200OK response message, stops playing to the television screen of the local terminal and releases television resources;

step five, the AP1 receives a 200OK response message of the AP2, sends an OK message to the UE1, stops playing to a local television screen and releases television resources;

step six, the UE1 receives the OK response message and sends a video call upgrade request to the UE2 through a control signaling connection;

step seven, the UE1 and the UE2 upgrade from a voice call to a video call, which is not detailed here.

The above example switches videos on the UE1 and UE2 sides to the mobile phone screen for playing and upgrades the language conversation. However, in another example, the AP2 may simply forward the received video stream to the UE1 for playing, without changing the playing mode on the UE2 side. In this other example, the step of interaction between AP1 and AP2 may be eliminated or the voice call may not be upgraded.

In a video call, a user may also pause or stop playing, and the process includes the following steps:

step one, a user 1 selects to pause playing through a UE 1;

step two, the UE1 sends a Pause request (Pause request) message to the AP 1;

step three, the AP1 interrupts the media stream transmission, sends an OK message to the UE1, and the television player pauses playing;

step four, the UE receives the OK response message and displays video playing pause on the interface;

step five, the user selects to stop playing through the UE 1;

step six, the UE sends a clear request (Teardown request) message to the AP 1;

step seven, the AP1 receives the control information transmitted by the UE and sends the control information to the AP 2;

step eight, after receiving the Teardown request message, the AP2 sends a 200OK response message, and stops the media stream of the local terminal from releasing resources;

step nine, the AP1 receives the 200OK response message of the AP2, sends an OK message to the UE1, and the home player stops playing to release resources;

step ten, the UE1 receives the OK response message, sends SIP BYE to the UE2, releases the resource, and the interface displays the end of the call;

step eleven, the UE2 returns a 200OK response message, the resource is released, and the interface displays that the call is finished.

Example four

The present embodiment provides a user terminal, as shown in fig. 12, including a memory 900, a processor 800, and a computer program stored on the memory 900 and executable on the processor 800, where the processor 800 implements the processing of any method executed by the user terminal according to the embodiment of the present invention when executing the computer program.

The embodiment also provides an access point device of a near field communication network, which includes a memory 900, a processor 800 and a computer program stored on the memory 900 and executable on the processor 800, and when the processor 800 executes the computer program, the processor 800 implements the processing of any method executed by an external device according to the embodiment of the present invention.

The present embodiment further provides an AP device, which mainly includes:

the first unit is used for receiving a media description query request sent by User Equipment (UE) and returning media description information of multimedia equipment connected with the AP equipment to the UE according to the media description query request;

and a second unit, configured to, when the UE communicates with a remote device using a multimedia function of the AP device, acquire the voice media stream and/or the video media stream by using the AP device, transmit the acquired voice media stream and/or video media stream to the remote device, receive the voice media stream and/or video media stream of the remote device, and transmit the acquired voice media stream and/or video media stream to each convergence terminal by using the AP device and play the stream to a user.

Optionally, in the above apparatus, the multimedia apparatus is an apparatus with a near field communication capability.

The invention also discloses a user UE terminal, which mainly comprises:

a first unit that acquires media description information of an AP device;

and a second unit, configured to send a first signaling to a registration server by using the media description information of the AP device when the UE device communicates with a remote device, and after the UE device receives a response of the first signaling, transmit a voice and/or video media stream to the remote device through the AP device, receive the voice and/or video media stream through the AP device, and play the voice and/or video media stream sent by the remote device through a multimedia device managed by the AP device.

Optionally, in the foregoing device, the first signaling includes one or more of the following:

call request, media modification request, INVITE request, re-INVITE request, UPDATE data UPDATE.

Optionally, in the foregoing device, the acquiring, by the first unit, the media description information of the multimedia device includes:

the first unit sends a media description query request to the AP equipment, and acquires media description information of the multimedia equipment managed by the AP equipment from a response returned by the AP equipment.

Optionally, the apparatus further comprises:

and a third unit, configured to send a first signaling to the registration server by using the media description information of the AP device when the UE device communicates with the remote device, and send a second signaling to the registration server by using a change of the media description information of the AP device, so that the UE device updates a multimedia function and interacts with the remote device.

Optionally, in the foregoing device, the second signaling includes one or more of the following:

media modification request, re-INVITE request, UPDATE data UPDATE.

As can be seen from the above embodiments, the solution of the embodiments of the present application has one or more of the following advantages:

1) the WLAN and the existing 3GPP network are fused to realize joint transmission, the 3GPP network is responsible for signaling connection, and the WLAN is responsible for data transmission, so that the purposes of load distribution and network performance improvement are achieved. The UE only needs to transmit the signaling, and the media data is transmitted through the external equipment, so that the data flow burden of the UE is greatly reduced, and the bandwidth is increased.

2) The terminal device manages external devices such as an external large screen, an external microphone and a high-definition camera through the external device (such as an AP hot spot). External multimedia equipment such as external high-quality microphone speaker, high definition digtal camera are fully utilized to carry out multimedia acquisition and broadcast. Playing the obtained voice and/or video media stream to a user by utilizing a multimedia function except the UE, and transmitting the obtained voice and/or video media stream to the remote equipment; a better user experience is obtained.

3) The terminal device distributes a part of calculation tasks (such as data transmission, coding and decoding and the like) to the external device for completion, so that the calculation burden is reduced, the power consumption is reduced, and the user experience is improved.

It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

Claims

1. A method of data transmission, comprising:

the first user terminal uses external hardware resources connected with external equipment of the terminal and indicates the external equipment of the terminal to carry out data transmission through the data service connection;

wherein the external device is an access point device; the external hardware resource connected by the first user terminal through the external equipment of the terminal comprises:

and the first user terminal instructs external equipment of the terminal to distribute the data between the first user terminal and the second user terminal to the first user terminal or the external hardware resource.

2. The method of claim 1, wherein:

the signaling connection is the signaling connection of a mobile communication service, and the data transmission is the data transmission in the mobile communication service; the external device is an access point device of a near field communication network.

3. The method of claim 1, wherein:

the data transmission comprises any one or more of the following:

4. The method of claim 3, wherein:

before the first user terminal instructs the external device of the terminal to perform data transmission through the data service connection, the method further includes: the first user terminal determines options to be applied from one or more of the following options according to a user instruction and indicates external equipment of the first user terminal:

5. The method of claim 1, wherein prior to the first user terminal establishing a signaling connection with the second user terminal, the method further comprises:

the first user terminal acquires information of external equipment of the first user terminal through a near field communication technology, wherein the acquired information of the external equipment of the first user terminal comprises one or more of capability description information and a communication identification address; the capability description information of the external device of the terminal comprises multimedia description information of the multimedia device connected with the external device of the terminal.

6. The method according to any of claims 1-5, wherein the first user terminal sends information of the external device of the terminal to the second user terminal in one or more of the following ways:

in a first mode, the first user terminal sends a control signaling to the second user terminal through the signaling connection, and a reserved field of the control signaling carries information of external equipment of the first user terminal;

in a second mode, the first user terminal sends a data packet to the second user terminal by using a transmission channel established after the transmission of the control signaling is finished, wherein the data packet carries information of the external equipment of the first user terminal.

7. A method of resource sharing, comprising:

the user terminal receives the information of the external equipment returned by the external equipment, and uses external hardware resources connected with the external equipment according to the information of the external equipment;

wherein the external device is an access point device; the external hardware resource connected by the user terminal by using the external equipment comprises:

and the user terminal instructs the external equipment to shunt the data between the user terminal and the remote equipment to the user terminal or the external hardware resource.

8. The method of claim 7, wherein:

the information of the external device comprises one or more of capability description information and a communication identification address of the external device; wherein the capability description information of the external device includes one or more of media description information of a multimedia device connected to the external device and operation capability information of a logical operation device connected to the external device.

9. The method of claim 7, wherein:

the external device includes an access point device of a near field communication network.

10. The method of any of claims 7-9, wherein:

the user terminal uses peripheral external hardware resources according to the information of the external equipment, and the method comprises the following steps:

the user terminal accesses the network server through external equipment;

11. The method of any of claims 7-9, wherein:

12. The method of any of claims 7-9, wherein:

13. A method of resource sharing, comprising:

the external equipment acquires resource information of external hardware resources connected with the external equipment, and returns the resource information of the external hardware resources to the user terminal as capability description information of the external equipment so that the user terminal can use the external hardware resources;

the external equipment distributes data between the user terminal and the remote equipment to the user terminal or the external hardware resource according to the user terminal indication; wherein the external device is an access point device.

14. The method of claim 13, wherein:

the external equipment is access point equipment of a short-distance communication network; the resource information of the external hardware resource comprises one or more of media description information of the multimedia device and arithmetic capability information of the logic arithmetic device.

15. The method of claim 13, wherein:

before or at the same time of returning the capability description information to the user terminal, the external device also returns the communication identification address of the external device to the user terminal.

16. The method of claim 13, wherein:

the information request is in the form of a data frame, and the data frame carries a beacon of a control signaling requesting the capability description information.

17. The method of claim 13, wherein:

after the external device returns the capability description information of the external hardware resource to the user terminal, the method further includes:

and the external equipment establishes the data service connection of the remote equipment according to the indication of the user terminal and transmits data through the data service connection.

18. The method of claim 17, wherein:

the remote end equipment is remote end external equipment which is connected with the user terminal through a remote end terminal called by mobile communication through near field communication;

the external device performs data transmission through the data service connection, and the data transmission includes:

19. The method of claim 17, wherein:

the data transmission comprises any one or more of the following:

sending an audio stream collected by the user terminal or the multimedia device connected with the external device;

sending a video stream collected by the user terminal or the multimedia equipment connected with the external equipment;

sending a file stored in the user terminal or the storage device connected with the external device;

receiving audio stream, and forwarding the audio stream to a user terminal or multimedia equipment connected with the external equipment for playing;

receiving the video stream, and forwarding the video stream to a user terminal or multimedia equipment connected with the external equipment for playing;

and receiving the file, and forwarding the file to a user terminal or a storage device connected with the external device for storage.

20. The method of claim 17, wherein:

before the external device establishes a data service connection with a remote device communicating with the user terminal according to an instruction of the user terminal, the method further includes: the external equipment receives an encryption key sent by the user terminal through near field communication, wherein the encryption key is generated in the process of authentication with a network side after the user terminal initiates a call request;

the external device performs data transmission through the data service connection, and the data transmission includes: and the external equipment encrypts the data to be transmitted by adopting the encryption key.

21. The method of claim 13, wherein:

22. A user terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the process of the method of any one of claims 1-12 when executing the computer program.

23. An access point device of a near field communication network comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the process of the method according to any of claims 13-21 when executing the computer program.

24. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the processing of the method according to any one of claims 1-21.