CN114679486B

CN114679486B - Communication interaction method and system, application terminal, client terminal and transfer server terminal

Info

Publication number: CN114679486B
Application number: CN202210323937.7A
Authority: CN
Inventors: 袁斯超; 黄万飞; 胡哲婷; 刘柏
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2022-03-29
Filing date: 2022-03-29
Publication date: 2024-05-07
Anticipated expiration: 2042-03-29
Also published as: CN114679486A

Abstract

The application provides a communication interaction method and system, an application end, a client end and a transfer server end. The method comprises the following steps: determining whether an available application instance exists or not in response to receiving an instance control request from a client forwarded by a transit server; in response to the existence of the available application instance, sending an identifier of a target application instance in the application instances to be distributed to a transit server to establish a mapping relationship between the target application instance and the client; responding to the received descriptive text session request from the client forwarded by the transfer server, generating response information for the descriptive text session request according to the mapping relation, and forwarding the response information to the client through the transfer server so as to establish a data channel between a target application instance in the application terminal and the client; and sending the media data of the target application instance to the client through the data channel, and transmitting the instruction data sent by the client to the target application instance through the data channel.

Description

Communication interaction method and system, application terminal, client terminal and transfer server terminal

Technical Field

The present application relates to the field of communications technologies, and in particular, to a communication interaction method and system, an application end, a client, and a relay server.

Background

In the related art, a method of remotely connecting a real-time authoring platform is generally to run the real-time authoring platform (also referred to as Unity) on a Windows system hosting platform, and a user connects to the hosting platform through a remote desktop technology, so that the real-time authoring platform is indirectly operated on the hosting platform through a browser.

But is limited by the host platform, when the real-time authoring platform needs to run on the platform of other operating systems, the real-time authoring platform is limited by the compatibility of remote desktop technology, so that the problem that the real-time authoring platform cannot be connected and communicated across the operating systems and only one implementation authoring platform can be connected is caused.

Disclosure of Invention

In view of the above, the present application aims to provide a communication interaction method and system, an application terminal, a client terminal, and a transfer server terminal.

Based on the above object, in a first aspect, the present application provides a communication interaction method, which is applied to an application end, and includes:

Determining whether an available application instance exists or not in response to receiving an instance control request from a client forwarded by a transit server;

In response to the available application examples, sending identifiers of target application examples in the application examples to be distributed to the transit server side so as to establish a mapping relation between the target application examples and the client side;

Responding to the received descriptive text session request from the client forwarded by the transit server, generating response information for the descriptive text session request according to the mapping relation, and forwarding the response information to the client through the transit server so as to establish a data channel between the target application instance in the application and the client;

and sending the media data of the target application instance to the client through the data channel, and transmitting the instruction data sent by the client to the target application instance through the data channel.

In one possible implementation of the present invention,

After the determining whether the application instance is available, the method further comprises:

generating queuing notification information in response to the fact that the available application instance does not exist, and forwarding the queuing notification information to the client through the transit server; the queuing notification information is used for indicating the client to enter a queuing state;

The method further comprises the steps of:

and determining whether available application examples exist or not according to a preset time interval for the client in the queuing state.

In one possible implementation of the present invention,

The establishing a data channel between the target application instance in the application end and the client end further includes:

Acquiring second media configuration information of the application end, and sending the second media configuration information to the transfer server end; the second media configuration information comprises a media type of the application end, a media format of the application end and a media interface of the application end;

Determining whether the first media configuration information is at least partially the same as the second media configuration information or not in response to receiving the first media configuration information of the client forwarded by the transit server; the first media configuration information comprises a media type of the client, a media format of the client and a media interface of the client;

and establishing a data channel between the target application instance and the client in response to the first media configuration information being at least partially identical to the second media configuration information.

In one possible implementation of the present invention,

The method further comprises the steps of:

acquiring a client candidate application address according to the transfer server;

and addressing and connecting the client according to the client candidate application address so as to establish communication connection between the application end and the client.

In a second aspect, the present application provides a communication interaction method, applied to a client, including:

The method comprises the steps of sending a first verification code generated in advance to a transfer server to establish communication connection with the transfer server;

sending an instance control request to the transfer server;

Generating a description text request in response to receiving an identifier of a target application instance from an application terminal sent by the transit server terminal, and sending the description text request to the transit server terminal so as to establish a mapping relation between the target application instance and the client terminal;

Responding to the response information of the application terminal sent by the transfer server terminal, and establishing a data channel between the target application instance in the application terminal and the client terminal according to the response information;

transmitting instruction data to the target application instance through the data channel, and receiving media data of the target application instance through the data channel.

In one possible implementation of the present invention,

The method further comprises the steps of:

acquiring application end candidate application addresses according to the transfer server;

and addressing and connecting the application end according to the application end candidate application address so as to establish communication connection between the client end and the application end.

In one possible implementation of the present invention,

Acquiring first media configuration information of the client side, and sending the first media configuration information to the transfer server side; the first media configuration information comprises a media type of the client, a media format of the client and a media interface of the client;

Determining whether the first media configuration information is at least partially the same as the second media configuration information or not in response to receiving second media configuration information of the application terminal forwarded by the transit server terminal; the second media configuration information comprises a media type of the application end, a media format of the application end and a media interface of the application end;

And establishing a data channel between the first media configuration information and the application terminal in response to the first media configuration information being at least partially identical to the second media configuration information.

In one possible implementation of the present invention,

The method further comprises the steps of:

Setting a data flow label pointing to the media data in response to receiving the media data of the target application instance transmitted by the application end through the data channel;

the media data is shown.

In a third aspect, the present application provides a communication interaction method, applied to a transit server, including:

Responding to a first verification code and an instance control request sent by a client, establishing communication connection with the client according to the first verification code, and forwarding the instance control request to an application end;

Responding to the received identifier of the target application instance sent by the application end, and establishing a mapping relation between the target application instance and the client according to the identifier of the target application instance;

responding to a received descriptive text session request sent by the client, and forwarding the descriptive text session request to the application;

Responding to the received response information sent by the application terminal, and forwarding the response information to the client terminal;

And respectively sending the generated client candidate application address and the generated application candidate application address to the application terminal and the client terminal.

In one possible implementation of the present invention,

The transfer server side comprises: a first verification code library;

the method comprises the steps of responding to receiving a first verification code and an instance control request sent by a client, establishing communication connection with the client according to the first verification code, forwarding the instance control request to an application end, and further comprising:

Receiving a first verification code sent by the client;

Determining whether the authentication code exists in the first verification code library;

and in response to the first verification code existing in the first verification code library, establishing communication connection with the client, and sending the instance control request to the application end.

In one possible implementation of the present invention,

The determining whether the first verification code exists in the first verification code library further comprises:

transmitting a first re-authentication instruction to the client in response to the first authentication code not being present in the first authentication code library; the first re-verification instruction is used for indicating the client to regenerate the first verification code and sending the regenerated first verification code to the transfer server.

In one possible implementation of the present invention,

The start-up information includes: a second verification code; the transfer server side comprises: a second verification code library;

the response to receiving the identifier of the target application instance sent by the application end, establishing a mapping relationship between the target application instance and the client according to the identifier of the target application instance, and further comprising:

Determining whether the second verification code exists in the second verification code library or not according to the received identifier of the target application instance sent by the application end;

And in response to the second verification code existing in the second verification code library, establishing communication connection with the application end, and sending the identifier of the target application instance to the client end.

In one possible implementation of the present invention,

The determining whether the second verification code exists in the second verification code library or not in response to receiving the identifier of the target application instance sent by the application end further comprises:

Responding to the fact that the second verification code does not exist in the second verification code library, and sending a second re-verification instruction to the application terminal; the second re-verification instruction is used for indicating the application end to re-generate a second verification code and sending the re-generated second verification code to the transfer server.

In one possible implementation of the present invention,

The feedback information further includes: a universal unique identification code;

The method further comprises the steps of:

Binding the universal unique identification code with the first verification code to determine the association relation between the client and the application terminal.

In one possible implementation of the present invention,

The sending the feedback information to the client further includes:

and sending the feedback information to the client side which has the association relation with the application side.

In a fourth aspect, the present application provides a communication interaction system comprising:

an application terminal configured to implement the communication interaction method according to the first aspect;

A client configured to implement the communication interaction method as described in the second aspect;

And the transit server is configured to implement the communication interaction method according to the third aspect.

From the above, it can be seen that, in the communication interaction method and system, the application end, the client and the transfer server provided by the application, when an instance control request from the client forwarded by the transfer server is received, the application end determines whether available resource instances exist in a plurality of resource instances running on the application end, if so, the application end sends an identifier of a target application instance to the transfer server, thereby establishing a mapping relationship between the target application instance and the client; when the application end receives the description text request from the client end forwarded by the transfer server end, response information aiming at the description text session request is generated according to the mapping relation, and the response information is sent to the client end through the transfer server end. And further establishing a data channel between the target application instance in the application end and the client, sending the media data of the target application instance to the client through the data channel, and transmitting instruction data sent by the client to the target application instance through the data channel. By establishing communication connection between the client and the application end, the client can directly control the application instance on the application end, and the application instance can run on any platform, so that the limitation of the host platform on the application instance is eliminated, and the applicability of the application instance control is improved. And the application instance is operated at the application end, so that the client side is not only enabled to control a program such as one application instance, but is enabled to perform communication interaction between the end and the end with the application end, and a plurality of application instances can be operated on the application end, so that the client side can control the plurality of application instances, the utilization rate of resources is improved, and the use cost of application instance control is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the present application or related art, the drawings that are required to be used in the description of the embodiments or related art will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort to those of ordinary skill in the art.

Fig. 1 shows an exemplary flowchart of a communication interaction method applied to an application side according to an embodiment of the present application.

Fig. 2 shows an exemplary flow diagram for establishing a communication connection between a client, an application, and a staging server according to an embodiment of the application.

Fig. 3 is an exemplary flowchart of a communication interaction method applied to a client according to an embodiment of the present application.

Fig. 4 is an exemplary flowchart of a communication interaction method applied to a relay server according to an embodiment of the present application.

Fig. 5 shows an exemplary structural schematic diagram of a communication interaction system according to an embodiment of the present application.

Fig. 6 shows an exemplary structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The present application will be further described in detail below with reference to specific embodiments and with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present application more apparent.

It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present application should be given the ordinary meaning as understood by one of ordinary skill in the art to which the present application belongs. The terms "first," "second," and the like, as used in embodiments of the present application, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.

As described in the background section, in the related art, a method of remotely connecting a real-time authoring platform is generally to run the real-time authoring platform (also referred to as Unity) on a Windows system hosting platform, and a user connects to the hosting platform through a remote desktop technology, so that the real-time authoring platform is indirectly operated on the hosting platform through a browser.

The applicant found through research that, in the related art, multiple Unity instances (i.e., implementation authoring platform instances) cannot be run on the same host platform, which is limited to the host platform. And when the Unity instance needs to be run on other platforms (such as a MacOS platform), the compatibility of the provided remote desktop technology is limited, remote service cannot be provided in time, and the problem that the related technology cannot connect communication with the Unity instance across an operating system and only can connect with one Unity instance is caused.

In this way, according to the communication interaction method and system, the application terminal, the client and the transfer server, when an instance control request forwarded by the transfer server from the client is received, the application terminal determines whether available resource instances exist in a plurality of resource instances running on the application terminal, if so, the application terminal sends an identifier of a target application instance to the transfer server, so that a mapping relation between the target application instance and the client is established; when the application end receives the description text request from the client end forwarded by the transfer server end, response information aiming at the description text session request is generated according to the mapping relation, and the response information is sent to the client end through the transfer server end. And further establishing a data channel between the target application instance in the application end and the client, sending the media data of the target application instance to the client through the data channel, and transmitting instruction data sent by the client to the target application instance through the data channel. By establishing communication connection between the client and the application end, the client can directly control the application instance on the application end, and the application instance can run on any platform, so that the limitation of the host platform on the application instance is eliminated, and the applicability of the application instance control is improved. And the application instance is operated at the application end, so that the client side is not only enabled to control a program such as one application instance, but is enabled to perform communication interaction between the end and the end with the application end, and a plurality of application instances can be operated on the application end, so that the client side can control the plurality of application instances, the utilization rate of resources is improved, and the use cost of application instance control is reduced.

The communication interaction method provided by the embodiment of the application is specifically described by a specific embodiment.

Referring to fig. 1, the communication interaction method applied to an application end provided by the embodiment of the present application specifically includes the following steps:

s102: and in response to receiving the instance control request from the client forwarded by the transit server, determining whether an available application instance exists.

S104: and in response to the available application examples, sending identifiers of target application examples in the application examples to be distributed to the transit server side so as to establish a mapping relation between the target application examples and the client side.

S106: and responding to the received descriptive text session request from the client forwarded by the transit server, generating response information for the descriptive text session request according to the mapping relation, and forwarding the response information to the client through the transit server so as to establish a data channel between the target application instance in the application and the client.

S108: and sending the media data of the target application instance to the client through the data channel, and transmitting the instruction data sent by the client to the target application instance through the data channel.

Referring to fig. 2, for step S102, the present application may be applied to a communication interaction system, which includes an application side, a client side, and a relay server side, where the relay server side may include a stun server side, a Websocket server side, and a dispatch server side. In this embodiment, the application may have a plurality of resource instances running therein, and the resource instances may be, for example, unity instances, that is, implementation authoring platform instances, and for convenience in describing embodiments of the present application, the description below uses the resource instances as Unity instances. When the application receives a resource instance control request sent by the client and forwarded by the scheduling server in the transit server, the application can prepare to allocate a Unity instance for the client. Because multiple Unity instances may be running on the application, the application may determine whether there is at least one available Unity instance from among the multiple Unity instances running on it to provide to the client.

For step S104, when the application determines that there is at least one available Unity instance in the multiple Unity instances running on the application, the identifier of the target application instance may be sent to the transit server, so as to establish a communication connection between the application and the Websocket server in the transit server. And the feedback information can also comprise starting information of at least one Unity instance in a plurality of Unity instances running on the application end, so that the client is informed that the controllable Unity instance is started and is in a state to be controlled.

The resource instance control request from the client may include corresponding information of the target Unity instance to be controlled, which is specified by the client, for example, an ID or an identification code of the target Unity instance. The application end can determine whether the target Unity instance running on the application end is available, if so, the application end sends feedback information carrying the starting information of the target Unity instance to the Websocket server end in the transit server end, and further the application end and the Websocket server end in the transit server end establish communication connection.

In some embodiments, if the application determines that there is no available Unity instance on it, for example, at this time, because the failure cannot start any one of the Unity instances or the target Unity instance, or no idle available Unity instances are available for the client to control, or the number of remaining available Unity instances does not satisfy the number of Unity instances that the client requests to control in the resource control request, queuing notification information is generated, and the application forwards the queuing notification information to the client through the transit server. The queuing notification information may be used to instruct the client to enter a queuing state, until the Unity instance available on the application can satisfy the number in the resource instance control request or the designated target Unity instance.

It may be appreciated that, according to a preset time interval, whether the Unity instances available on the application end can meet the number or the specified target Unity instances in the resource instance control request may be determined at intervals, and if the Unity instances available on the application end can meet the number or the specified target Unity instances in the resource instance control request, feedback information may be sent to the Websocket server in the transit server. And, with the start of the Unity instance, the application end can generate a string of random uuid (also called universal unique identification code) character strings, so as to establish communication connection with the Websocket server in the transit server.

For step S106, when the application receives a description text session request (Offer SDP) forwarded by the Websocket server from the client, the rtcp connection method may be called to perform media negotiation, and then the rtcp connection method may be called to create response information of the application according to the received description text session request, where the response information includes a media description (ANSWER SDP) of the application, and then the response information is sent to the client through the Websocket server in the transit server.

Referring to fig. 2, in some embodiments, an application may access a stun server in a transit server, and obtain an application candidate application address corresponding to the application by sending a Binding Request (Binding Request) to the stun server (ICE CANDIDATE). The stun server may allow all NAT (network address translator) clients (e.g., computers behind a firewall) or application terminals to make communication connection with application terminals or clients outside the local area network. Through the stun server, the client or application can acquire its own candidate application address (i.e., public address), the NAT type blocked in front of it, and the internet-side port connected to a particular local port through NAT. This information will be used to establish UDP (user datagram protocol) communication between the client and the application in order to enable a communication connection between the two.

Referring to fig. 2, after the application terminal obtains the application terminal candidate application address of the application terminal, the application terminal candidate application address is sent to the client terminal through the Websocket server terminal, so that the client terminal can establish communication connection with the application terminal according to the application terminal candidate application address.

In some embodiments, in order to enable the application end and the client end to establish a communication connection, while the application end sends the application end candidate application address to the client end through the Websocket server end, the application end can also receive the client end candidate application address sent by the client end through the Websocket server end.

In some embodiments, the application may establish a communication connection with the client based on the client candidate application address, e.g., the application may address to connect the client based on the client candidate application address, thereby establishing a communication connection between the application and the client.

Further, the application end can acquire second media configuration information of the application end, and send the second media configuration information to the Websocket server end, and the Websocket server end waits for forwarding the first media configuration information from the client end sent by the client end. The second media configuration information may include a media type of the application end, a media format of the application end, and a media interface of the application end; the first media configuration information may include a media type of the client, a media format of the client, and a media interface of the client. When the application receives the first media configuration information, it is further determined whether the first media configuration information is partially the same as the second media configuration information, for example, the media formats in both include the MP3 format of audio. After determining that the first media configuration information is at least partially the same as the second media configuration information, a data channel between the application and the client may be established.

Still further, after the communication connection and the data channel between the application end and the client end are established, the Unity instance waiting application end in the state to be controlled receives a control instruction from the client end. When the application receives the control instruction sent by the client, at least one Unity instance in a state to be controlled can be controlled by the control instruction of the client.

In some embodiments, when the application receives an instance control instruction of the client, the application may send a data stream of the Unity instance controlled by the control instruction to the client through the media channel, so that the data stream may be played and shown on the client. The data stream may be an audio/video stream.

Referring to fig. 3, the communication interaction method applied to the client provided by the embodiment of the application specifically includes the following steps:

s302: and sending the pre-generated first verification code to the transit server to establish communication connection with the transit server.

S304: and sending an instance control request to the transit server.

S306: and generating a description text request in response to receiving an identifier of a target application instance from an application terminal sent by the transit server terminal, and sending the description text request to the transit server terminal so as to establish a mapping relation between the target application instance and the client terminal.

S308: and responding to the received response information of the application terminal sent by the transfer server terminal, and establishing a data channel between the target application instance in the application terminal and the client terminal according to the response information.

S310: transmitting instruction data to the target application instance through the data channel, and receiving media data of the target application instance through the data channel.

For step S302, the client may generate the first verification code in advance, and send the first verification code to the Websocket server in the transit server, so as to establish communication connection with the transit server. Wherein the first authentication code may be a 6-bit random authentication code.

In some embodiments, the client may establish a communication connection with a Websocket server in the transit server by placing the first validation code in a Uniform Resource Locator (URL).

For step S304, when the client wants to control the Unity instance of the application end, the client may send an instance control request to the Websocket server, and then the Websocket server forwards the instance control request to the scheduling server, so that the scheduling server may request the application end for scheduling of the Unity instance according to the instance control request.

For step S306, when the client receives the identifier of the target application instance and the feedback message forwarded by the Websocket server from the application end, and reads the start information of at least one of the plurality of Unity instances of the application end carried in the feedback message, a description text session request may be generated, and the description text session request is sent to the application end through the Websocket server.

Referring to fig. 2, in some embodiments, after receiving the identifier of the target application instance, the client creates peerConnection locally, and then invokes the rtcp connection. Createoff method to generate a description text session request of the client, which may specifically be a media description Offer SDP, and sends the description text session request to the application through the Websocket server.

For step S308, since the application may generate the response information, that is, the media description ANSWER SDP, according to the received description text session request and send the response information to the client through the Websocket server, when the client receives the response information sent by the application forwarded by the Websocket server, the application candidate application address sent by the application may be obtained from the Websocket server.

Referring to fig. 2, after the client obtains the client candidate application address of the client, the client candidate application address is sent to the application end through the Websocket server, so that the application end can establish communication connection with the client according to the client candidate application address.

Referring to fig. 2, in some embodiments, a client may access a stun server in a transit server, and obtain a client candidate application address corresponding to the client by sending a Binding Request (ICE CANDIDATE) to the stun server. The stun server may allow all NAT (network address translator) clients (e.g., computers behind a firewall) or application terminals to make communication connection with application terminals or clients outside the local area network. Through the stun server, the client or application can acquire its own candidate application address (i.e., public address), the NAT type blocked in front of it, and the internet-side port connected to a particular local port through NAT. This information will be used to establish UDP (user datagram protocol) communication between the client and the application in order to enable a communication connection between the two.

In some embodiments, the client may establish a communication connection with the application end in an addressing manner according to the obtained candidate application address of the application end, so as to control a Unity instance on the application end, which corresponds to the client and enters a to-be-controlled state.

Further, the client may acquire the first media configuration information of the client, and send the first media configuration information to the Websocket server, and wait for the Websocket server to forward the second media configuration information from the application terminal sent by the application terminal. The first media configuration information may include a media type of the client, a media format of the client, and a media interface of the client; the second media configuration information may include a media type of the application side, a media format of the application side, and a media interface of the application side. When the client receives the second media configuration information, it is further determined whether the first media configuration information is partially the same as the second media configuration information, for example, the media formats in both the first and second media configuration information include the MP3 format of the audio. After determining that the first media configuration information is at least partially identical to the second media configuration information, a media data transmission channel between the application and the client may be established.

Still further, after establishing the communication connection between the application end and the client end and the media data transmission channel, the client end may send a control instruction to the Unity instance in a state to be controlled. When the application receives the control instruction sent by the client, at least one Unity instance in a state to be controlled can be controlled by the control instruction of the client.

In some embodiments, when the application end receives the resource instance control instruction of the client end, the data stream of the Unity instance controlled by the control instruction can be sent to the client end through the media channel, and after the client end receives the data stream sent by the application end, the data stream label can be set for pointing to the data stream, and further, the data stream is displayed. The data stream label can be a video label, wherein srcObejct attributes point to the data stream, the data stream can be an audio/video stream, and when the data stream is the audio/video stream, the client can call a play method to play the audio/video stream, so that the purpose of displaying the data stream is achieved.

Referring to fig. 4, the communication interaction method applied to the transit server provided by the embodiment of the application specifically includes the following steps:

S402: and responding to the received first verification code and the instance control request sent by the client, establishing communication connection with the client according to the first verification code, and forwarding the instance control request to the application end.

S404: and responding to the received identifier of the target application instance sent by the application end, and establishing a mapping relation between the target application instance and the client according to the identifier of the target application instance.

S406: and responding to the received descriptive text session request sent by the client, and forwarding the descriptive text session request to the application.

S408: and forwarding the response information to the client in response to receiving the response information sent by the application terminal.

S410: and respectively sending the generated client candidate application address and the generated application candidate application address to the application terminal and the client terminal.

For step S402, after receiving the first verification code and the instance control request sent by the client by the Websocket server in the transit server, a communication connection with the client may be established according to the first verification code, for example, the first verification code is verified, if the identity of the client is verified to be legal, a communication connection may be established with the client, further, the instance control request may be sent to the dispatch server, and the dispatch server forwards the instance control request to the application end to initiate a call for invoking the Unity instance.

In some embodiments, the Websocket server in the transit server may be provided with a first verification code library, the Websocket server receives a first verification code, such as a random authentication code, sent by the client, the Websocket server may further determine that the first verification code exists in the first verification code library, if the first verification code exists in the first verification code library, a communication connection between the Websocket server and the client may be established, and further request idle Unity from the dispatch service, and send an instance control request to the application through the dispatch service, thereby requesting to invoke a Unity instance in the application.

It should be noted that, if the Websocket server determines that the first verification code does not exist in the first verification code library, the first re-verification instruction may be sent to the client, and after the client receives the first re-verification instruction, the client may regenerate a new first verification code under the instruction of the first re-verification instruction, and send the regenerated new first verification code to the Websocket server for re-verification.

It can be understood that, besides verifying by using the authentication code, the Websocket server can also verify whether the account password input by the client exists in a password library pre-stored by the Websocket server by inputting the account password to the Websocket server, if so, the identity verification of the client is legal, if not, a first re-verification instruction can be sent to the client, and after the client continuously inputs the account password which cannot pass the verification five times, the client is frozen so that the client cannot be connected to the Websocket server, thereby ensuring the reliability and the security of the communication connection between the client and the Websocket server.

For step S404, after the Websocket server in the transit server receives the identifier of the target application instance and the feedback information sent by the application end, a communication connection with the application end may be established according to the identifier of the target application instance, and the feedback information may be further sent to the client. It should be noted that, the feedback information may include startup information of at least one Unity instance in the multiple Unity instances at the application end.

In some embodiments, the feedback information sent by the application end may include a second verification code for verifying the identity of the application end, and the Websocket server is correspondingly provided with a second verification code library. The Websocket server receives a second verification code, such as a random authentication code, sent by the application end, and the Websocket server can further determine that the second verification code exists in a second verification code library, if the second verification code exists in the second verification code library, communication connection between the Websocket server and the application end can be established, and further feedback information is forwarded to the client.

It should be noted that, if the Websocket server determines that the second verification code does not exist in the second verification code library, the Websocket server may send a second verification instruction to the application end, and after receiving the second verification instruction, the application end may regenerate a new second verification code under the instruction of the second verification instruction, and send the regenerated new second verification code to the Websocket server for verification again.

It can be understood that, besides verifying by using the authentication code, the Websocket server can also verify whether the account password input by the application end exists in a password library prestored by the Websocket server by inputting the account password to the Websocket server, if so, the identity verification of the application end is legal, if not, a second re-verification instruction can be sent to the application end, and after the application end continuously inputs the account password which cannot pass the verification for five times, the application end is frozen so that the application end cannot be connected to the Websocket server, thereby ensuring the reliability and the security of the communication connection between the application end and the Websocket server.

In some embodiments, the feedback information sent by the application end may further include a universal unique identifier, and along with the start of the Unity instance, the application end may generate a string of random uuid (also referred to as a universal unique identifier) strings, so as to establish a communication connection with a Websocket server in the transit server. The Websocket server can bind the universal unique identification code of the application terminal with the first verification code of the client terminal, so that the association relation between the client terminal and the application terminal is determined, the client terminal and the application terminal which establish the association relation correspond to each other, and the application terminal which has the association relation with the client terminal can respond in time when the client terminal initiates a request to the application terminal.

In some embodiments, when the Websocket server sends feedback information to the client, the feedback information or other information sent to the application may be sent to the application that has an association relationship with the client based on the association relationship between the client and the application.

For step S406, when the Websocket server receives the description text session request sent by the client, that is, the media description Offer SDP, the description text session request may be forwarded to the application end.

For step S408, when the Websocket server receives the response information sent by the application, where the response information may include the media description ANSWER SDP of the application, the response information may be forwarded to the client.

In some embodiments, after the client obtains the client candidate application address at the stun server, the client candidate application address is forwarded to the application end through the Websocket server, and after the application end obtains the application candidate application address at the stun server, the application candidate application address is forwarded to the client through the Websocket server.

It should be noted that, the method of the embodiment of the present application may be performed by a single device, for example, a computer or a server. The method of the embodiment can also be applied to a distributed scene, and is completed by mutually matching a plurality of devices. In the case of such a distributed scenario, one of the devices may perform only one or more steps of the method of an embodiment of the present application, the devices interacting with each other to accomplish the method.

It should be noted that the foregoing describes some embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Based on the same inventive concept, the application also provides a communication interaction system corresponding to the method of any embodiment.

Referring to fig. 5, the communication interaction system includes: the system comprises an application terminal, a client terminal and a transfer server terminal; wherein,

An application end configured to

Transmitting the media data of the target application instance to the client through the data channel, and transmitting the instruction data transmitted by the client to the target application instance through the data channel;

A client configured to

sending an instance control request to the transfer server;

transmitting instruction data to the target application instance through the data channel, and receiving media data of the target application instance through the data channel;

a transit server configured to

In one possible implementation, the application end is further configured to:

The method further comprises the steps of:

In one possible implementation, the application end is further configured to:

In one possible implementation, the client is further configured to:

the media data is shown.

In one possible implementation manner, the transit server includes: a first verification code library;

The transit server is further configured to:

Receiving a first verification code sent by the client;

In one possible implementation, the transit server is further configured to:

In one possible implementation, the start information includes: a second verification code; the transfer server side comprises: a second verification code library;

The transit server is further configured to:

In one possible implementation, the transit server is further configured to:

In one possible implementation manner, the feedback information further includes: a universal unique identification code;

The transit server is further configured to:

In one possible implementation, the transit server is further configured to:

For convenience of description, the above system is described as being functionally divided into various modules, respectively. Of course, the functions of each module may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.

The system of the foregoing embodiment is configured to implement the corresponding communication interaction method applied to the application side in any of the foregoing embodiments, or the corresponding communication interaction method applied to the client side in any of the foregoing embodiments, or the corresponding communication interaction method applied to the transit server side in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein again.

Based on the same inventive concept, the application also provides an electronic device corresponding to the method of any embodiment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor implements the communication interaction method applied to the application end described in any embodiment, the communication interaction method applied to the client end described in any embodiment, or the communication interaction method applied to the transit server end described in any embodiment when executing the program. Fig. 6 shows a more specific hardware architecture of an electronic device according to this embodiment, where the device may include: processor 610, memory 620, input/output interface 630, communication interface 640, and bus 650. Wherein processor 610, memory 620, input/output interface 630, and communication interface 640 enable communication connections among each other within the device via bus 650.

The processor 610 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an Application SPECIFIC INTEGRATED Circuit (ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solutions provided in the embodiments of the present disclosure.

The Memory 620 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage, dynamic storage, etc. Memory 620 may store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented in software or firmware, relevant program codes are stored in memory 620 and invoked for execution by processor 610.

The input/output interface 630 is used for connecting with an input/output module to realize information input and output. The input/output module may be configured as a component in a device (not shown in the figure) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.

The communication interface 640 is used to connect a communication module (not shown in the figure) to enable communication interaction between the present device and other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).

Bus 650 includes a path to transfer information between components of the device (e.g., processor 610, memory 620, input/output interface 630, and communication interface 640).

It should be noted that although the above device only shows the processor 610, the memory 620, the input/output interface 630, the communication interface 640, and the bus 650, in the implementation, the device may further include other components necessary for achieving normal operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.

The electronic device of the foregoing embodiment is configured to implement the corresponding communication interaction method applied to the application side in any of the foregoing embodiments, or the corresponding communication interaction method applied to the client side in any of the foregoing embodiments, or the corresponding communication interaction method applied to the transit server side in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein again.

Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the application (including the claims) is limited to these examples; the technical features of the above embodiments or in the different embodiments may also be combined within the idea of the application, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the application as described above, which are not provided in detail for the sake of brevity.

Additionally, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown within the provided figures, in order to simplify the illustration and discussion, and so as not to obscure the embodiments of the present application. Furthermore, the devices may be shown in block diagram form in order to avoid obscuring the embodiments of the present application, and also in view of the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the present application are to be implemented (i.e., such specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the application, it should be apparent to one skilled in the art that embodiments of the application can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative in nature and not as restrictive.

While the application has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of those embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

The present embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, and the like, which are within the spirit and principles of the embodiments of the application, are intended to be included within the scope of the application.

Claims

1. The communication interaction method is characterized by being applied to an application end and comprising the following steps of:

In response to the available application examples, sending identifiers and feedback information of target application examples in the application examples to be distributed to the transit server so as to establish a mapping relation between the target application examples and the client; wherein the feedback information includes: a second verification code; the transfer server side comprises: a second verification code library; the responding to the available application examples, sending the identifier of the target application example in the application examples to be distributed and the feedback information to the transit server side so as to establish the mapping relation between the target application example and the client side, and the method comprises the following steps:

In response to the available application examples, sending identifiers and feedback information of target application examples in the application examples to be distributed to the transit server side so that the transit server side determines whether the second verification code exists in the second verification code library;

Responding to the second verification code existing in the second verification code library, establishing communication connection with the transfer server, and sending the identifier of the target application instance to the client through the transfer server so as to establish a mapping relation between the target application instance and the client;

2. The method of claim 1, wherein after determining whether there is an application instance available, further comprising:

The method further comprises the steps of:

3. The method of claim 1, wherein the establishing a data channel between the target application instance in the application end and the client end further comprises:

4. The method according to claim 1, characterized in that the method further comprises:

5. A communication interaction method, applied to a client, comprising:

sending an instance control request to the transfer server;

Generating a description text request in response to receiving an identifier of a target application instance from an application terminal sent by the transit server terminal, and sending the description text request to the transit server terminal so as to establish a mapping relation between the target application instance and the client terminal; the identifier of the target application instance is a communication connection between the transfer server and the application end is established to be sent to the client when the second verification code in the feedback information is determined to exist in a second verification code library in the transfer server according to the received identifier of the target application instance and feedback information sent by the application end;

6. The method according to claim 5, further comprising:

7. The method of claim 5, wherein the establishing a data channel between the target application instance in the application end and the client end further comprises:

8. The method according to claim 5, further comprising:

the media data is shown.

9. The communication interaction method is characterized by being applied to a transit server and comprising the following steps of:

In response to receiving an identifier of a target application instance and feedback information sent by the application end, establishing a mapping relationship between the target application instance and the client according to the identifier of the target application instance and the feedback information; wherein the feedback information includes: the second verification code, the transfer server side includes: a second verification code library; the response to receiving the identifier of the target application instance and feedback information sent by the application end, the mapping relationship between the target application instance and the client end is established according to the identifier of the target application instance and the feedback information, including:

Determining whether the second verification code exists in the second verification code library or not according to the received identifier of the target application instance and feedback information sent by the application end;

Responding to the second verification code existing in the second verification code library, establishing communication connection with the application end, and sending the identifier of the target application instance to the client end;

10. The method of claim 9, wherein the transit server comprises: a first verification code library;

Receiving a first verification code sent by the client;

Determining whether the first verification code exists in the first verification code library;

11. The method of claim 10, wherein the determining whether the first captcha exists in the first captcha library further comprises:

12. The method of claim 9, wherein the determining whether the second verification code exists in the second verification code library in response to receiving the identifier of the target application instance sent by the application end, further comprises:

13. The method of claim 9, wherein the feedback information further comprises: a universal unique identification code;

The method further comprises the steps of:

14. The method of claim 13, wherein the sending the feedback information to the client further comprises:

15. A communication interactive system, comprising:

An application configured to implement the method of any one of claims 1 to 4;

a client configured to implement the method of any one of claims 5 to 8;

A transit server configured to implement the method of any one of claims 9 to 14.