CN114007143A

CN114007143A - Information interaction method, device and equipment, computer storage medium and program product

Info

Publication number: CN114007143A
Application number: CN202111271390.2A
Authority: CN
Inventors: 潘博渊
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-10-29
Filing date: 2021-10-29
Publication date: 2022-02-01

Abstract

The application provides an information interaction method, an information interaction device, information interaction equipment, a computer storage medium and a program product; the embodiment of the application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, vehicle-mounted and the like, and relates to the cloud computing technology; the method comprises the following steps: issuing an initial information stream aiming at the display equipment; wherein, the initial information flow is used as a reference when inputting an operation event; receiving a network data packet sent by input equipment, and analyzing an operation event triggered by an initial information flow from the network data packet; generating an update information stream in response to the operational event; and sending the updating information flow to the display equipment so that the display equipment displays the updating information flow. According to the application, the communication delay during the control of the application software can be reduced.

Description

Information interaction method, device and equipment, computer storage medium and program product

Technical Field

The present application relates to cloud computing technologies, and in particular, to an information interaction method, apparatus, device, computer storage medium, and program product.

Background

With the increasing requirements for display effects and interaction modes, more and more users can select to display the running results of the application software on the display equipment with more excellent display effects and select to operate on other input equipment so as to control the application software. For example, the picture of the virtual scene is displayed through the smart television, and the virtual scene is interacted through peripherals such as a handle and the like, so that the user experience is enhanced. In the related art, when the application software running on the server is controlled through the input device, the feedback content of the application software needs to be generated and issued after a period of time of user operation, that is, a problem of high communication delay exists for the control of the application software.

Disclosure of Invention

Embodiments of the present application provide an information interaction method, apparatus, device, computer-readable storage medium, and program product, which can reduce communication delay when controlling application software.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides an information interaction method, which comprises the following steps:

issuing an initial information stream aiming at the display equipment; wherein the initial information flow is used as a reference when an operation event is input;

receiving a network data packet sent by an input device, and analyzing the operation event triggered by the initial information flow from the network data packet;

generating an update information stream in response to the operational event;

and sending the updating information stream to the display equipment so that the display equipment displays the updating information stream.

in response to an input operation, determining an operation event aiming at an initial information stream being displayed by the display equipment; wherein the initial information flow is used as a reference when an operation event is input;

packaging the operation event to obtain a network data packet;

and sending the network data packet to a server so that the server generates an update information stream according to an operation event in the network data packet and pushes the update information stream to the display equipment.

receiving an initial information stream sent by a server; wherein the initial information flow is used as a reference when an operation event is input;

displaying the initial information flow on an information display interface;

receiving an update information stream sent by the server;

replacing the initial information flow of the information presentation interface with the updated information flow, wherein the updated information flow is generated by the server in response to an operation event parsed from a network data packet sent by an input device.

An embodiment of the present application provides an information interaction apparatus, including:

the first sending module is used for issuing an initial information stream aiming at the display equipment; wherein the initial information flow is used as a reference when an operation event is input;

the first receiving module is used for receiving a network data packet sent by input equipment and analyzing the operation event triggered by the initial information flow from the network data packet;

the information generation module is used for responding to the operation event and generating an update information stream;

the first sending module is further configured to send the update information stream to the display device, so that the display device displays the update information stream.

In some embodiments of the present application, the operational event comprises: an interaction event for interacting with a virtual object in a virtual scene; the information generation module is further used for responding to the interaction event and updating the virtual scene to obtain an updated virtual scene; rendering the updated virtual scene to obtain the updated information stream.

In some embodiments of the present application, the interaction event comprises: one or more of an event that controls an action of a virtual object in the virtual scene, an event that controls an appearance of the virtual object in the virtual scene, and an event that controls an exchange of the virtual object in the virtual scene.

In some embodiments of the present application, the first receiving module is further configured to receive a scene loading instruction sent by the display device; the scene loading instruction is used for triggering loading of the virtual scene;

the information generation module is further used for responding to the scene loading instruction and loading the virtual scene; rendering the virtual scene to obtain the initial information flow; the initial information stream includes the loaded pictures and sound effects of the virtual scene.

the event determining module is used for responding to the input operation and determining an operation event aiming at the initial information flow displayed by the display equipment; wherein the initial information flow is used as a reference when an operation event is input;

the data encapsulation module is used for encapsulating the operation event to obtain a network data packet;

and the second sending module is used for sending the network data packet to a server so that the server generates an update information stream according to an operation event in the network data packet and pushes the update information stream to the display equipment.

In some embodiments of the present application, the information interaction apparatus further includes: a second receiving module; the second receiving module is configured to receive the first routing information of the server sent by the display device;

the second sending module is further configured to send the network data packet to the server according to the first routing information.

In some embodiments of the present application, the information interaction apparatus further includes: a network connection module;

the second sending module is further configured to send an information acquisition request to the display device through a hardware connection with the display device;

the second receiving module is further configured to receive network connection information and second routing information sent by the display device for the information acquisition request; the network connection information at least comprises a network identifier and a password of a local area network where the display equipment is located;

the network connection module is used for accessing the local area network where the display equipment is located according to the network connection information;

and the second sending module is further configured to send a prompt message of successful network access to the display device according to the second routing information.

the third receiving module is used for receiving the initial information flow sent by the server; wherein the initial information flow is used as a reference when an operation event is input;

the information display module is used for displaying the initial information flow on an information display interface;

the third receiving module is further configured to receive an update information stream sent by the server;

the information presentation module is further configured to replace the initial information flow of the information presentation interface with the updated information flow, where the updated information flow is generated by the server in response to an operation event parsed from a network data packet sent by an input device.

In some embodiments of the present application, the information interaction apparatus further includes: a third sending module; the third sending module is configured to send a scene loading instruction to the server; the scene loading instruction is used for triggering loading of a virtual scene; and sending the first routing information of the server to the input equipment.

In some embodiments of the present application, the information interaction apparatus further includes: an information acquisition module;

the third receiving module is further configured to receive an information acquisition request sent by the input device;

the information acquisition module is used for responding to the information acquisition request, and acquiring network connection information and second routing information of the display equipment; the network connection information at least comprises a network identifier and a password of a local area network where the display equipment is located;

the third sending module is further configured to send the network connection information and the second routing information to the input device;

the third receiving module is further configured to receive a prompt message that the input device successfully accesses the network;

the information display module is also used for responding to the prompt message and displaying operation prompt information on a prompt interface so as to prompt the user to operate by using the input equipment.

An embodiment of the present application provides a server, including:

a first memory for storing executable instructions;

and the first processor is used for realizing the information interaction method on the server side provided by the embodiment of the application when the executable instructions stored in the first memory are executed.

An embodiment of the present application provides an input device, including:

a second memory for storing executable instructions;

and the second processor is used for realizing the information interaction method on the input device side provided by the embodiment of the application when the executable instructions stored in the second memory are executed.

An embodiment of the present application provides a display device, including:

a third memory for storing executable instructions;

and the third processor is used for realizing the information interaction method on the display equipment side provided by the embodiment of the application when the executable instructions stored in the third memory are executed.

An embodiment of the present application provides a computer-readable storage medium, which stores executable instructions, and is configured to cause a first processor to execute to implement an information interaction method provided by a server side in an embodiment of the present application, or to cause a second processor to execute to implement an information interaction method provided by an input device side in an embodiment of the present application, or to cause a third processor to execute to implement an information interaction method provided by a display device side in an embodiment of the present application.

An embodiment of the present application provides a computer program product, which includes a computer program or an instruction, where the computer program or the instruction, when executed by a first processor, implements the information interaction method on the server side provided in the embodiment of the present application, or when executed by a second processor, implements the information interaction method on the input device side provided in the embodiment of the present application, or when executed by a third processor, implements the information interaction method on the display device side provided in the embodiment of the present application

The embodiment of the application has the following beneficial effects: the input device directly sends the network data packet containing the operation event to the server, so that the time required for the display device to receive and process the network data packet can be reduced, the network data packet can be quickly sent to the server, the server quickly responds based on the operation event in the network data packet, an update information stream is generated, and the update information stream is directly sent to the display device for display. Therefore, the feedback content of the application software can be generated and issued in a short time after the user operates the application software, and the communication delay for controlling the application software is effectively reduced.

Drawings

FIG. 1 is a schematic diagram of forwarding operational instructions to a server through a display device;

FIG. 2 is a schematic diagram of a process for controlling a cloud game via an input device;

FIG. 3 is a schematic structural diagram of an information interaction system provided in an embodiment of the present application;

FIG. 4 is a schematic structural diagram of the server in FIG. 3 according to an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of the input device in FIG. 3 according to an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of the display apparatus in FIG. 3 provided by an embodiment of the present application;

fig. 7 is a first flowchart illustrating an information interaction method according to an embodiment of the present application;

fig. 8 is a flowchart illustrating a second information interaction method according to an embodiment of the present application;

fig. 9 is a third schematic flowchart of an information interaction method provided in an embodiment of the present application;

fig. 10 is a hardware deployment structure diagram of a running environment of a cloud game provided in an embodiment of the present application;

fig. 11 is a schematic diagram of a flow of a cloud game runtime provided in an embodiment of the present application;

FIG. 12 is a schematic diagram of a process for establishing communication between an input peripheral and a cloud gaming server according to an embodiment of the present application;

FIG. 13 is a diagram illustrating a process for accessing a wireless network by an input peripheral according to an embodiment of the present application;

fig. 14 is a schematic process diagram of an input peripheral device sending a notification of successful access to a smart tv according to an embodiment of the present application;

fig. 15 is a schematic process diagram of starting a cloud game according to an embodiment of the present application.

Detailed Description

In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the attached drawings, the described embodiments should not be considered as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first \ second \ third" are only to distinguish similar objects and do not denote a particular order, but rather the terms "first \ second \ third" are used to interchange specific orders or sequences, where appropriate, so as to enable the embodiments of the application described herein to be practiced in other than the order shown or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.

1) Cloud Computing (Cloud Computing) refers to a delivery and usage mode of an IT infrastructure, and refers to a network obtaining required resources in an on-demand and easily extensible manner; the generalized cloud computing refers to a delivery and use mode of a service, and refers to obtaining a required service in an on-demand and easily-extensible manner through a network. Such services may be IT and software, internet related, or other services. Cloud Computing is a product of development and fusion of traditional computers and Network Technologies, such as Grid Computing (Grid Computing), Distributed Computing (Distributed Computing), Parallel Computing (Parallel Computing), Utility Computing (Utility Computing), Network Storage (Network Storage Technologies), Virtualization (Virtualization), Load balancing (Load Balance), and the like.

With the development of diversification of internet, real-time data stream and connecting equipment and the promotion of requirements of search service, social network, mobile commerce, evolution cooperation and the like, cloud computing is rapidly developed. Different from the prior parallel distributed computing, the generation of cloud computing can promote the revolutionary change of the whole internet mode and the enterprise management mode in concept.

2) Cloud Gaming (Cloud Gaming), also known as game on Demand (Gaming on Demand), is an online Gaming technology based on Cloud computing technology. Cloud gaming technology enables light-end devices (Thin clients) with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud game scene, a game is not operated in a player game terminal but in a cloud server, and the cloud server renders the game scene into a video and audio stream which is transmitted to the player game terminal through a network. The player game terminal does not need to have strong graphic operation and data processing capacity, and only needs to have basic streaming media playing capacity and capacity of acquiring player input instructions and sending the instructions to the operation server.

3) In response to the condition or state on which the performed operation depends, one or more of the performed operations may be in real-time or may have a set delay when the dependent condition or state is satisfied; there is no restriction on the order of execution of the operations performed unless otherwise specified.

4) The virtual scene is a virtual scene displayed (or provided) on the terminal. The virtual scene may be a simulation environment for a real event, a semi-simulation semi-fictional virtual environment, or a pure fictional virtual environment. The virtual scene may be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The dimension of the virtual scene is not limited in the embodiment of the application. For example, the virtual scene may include sky, land, sea, virtual objects, and the like, the land may include environmental elements such as desert, city, and the like, and the user may control the virtual objects to move in the virtual scene.

5) Virtual objects, the appearance of various people and objects in the virtual scene that can interact, or movable objects in the virtual scene. The movable object may be a virtual character, a virtual animal, an animation character, etc., such as a character, an animal, a plant, an oil drum, a wall, a stone, etc., displayed in a virtual scene. The virtual object may be an avatar in the virtual scene that is virtual to represent the user. The virtual scene may include a plurality of virtual objects, each virtual object having its own shape and volume in the virtual scene and occupying a portion of the space in the virtual scene.

The virtual object may be a user Character controlled by an operation on the client, an Artificial Intelligence (AI) Character set in a virtual scene match through training, or a Non-user Character (Non-Player-Character, NPC) set in a virtual scene interaction. The virtual object may also be a virtual character that has antagonistic interactions in the virtual scene. The number of virtual objects participating in the interaction in the virtual scene may be preset or may be dynamically determined according to the number of clients participating in the interaction.

6) The input device is a hardware device which receives direct operation of a user to send out control instructions, such as a mouse, a keyboard, a handle and the like.

7) The display device is a hardware device, such as a television, a smart phone, a personal computer, etc., which receives input information from the input device and feeds back information such as sound and image to a user.

By running the application which needs complex data processing or powerful graphic computation on the cloud server and only using the terminal for displaying, the terminal with various performances can provide powerful functional services for users. With the increasing requirements for display effects and interaction modes, more and more users can select to display the running results of the application software on the display equipment with more excellent display effects and select to operate on other input equipment so as to control the application software. For example, the picture of the virtual scene is displayed through the smart television, and the virtual scene is interacted through peripherals such as a handle and the like, so that the user experience is enhanced.

In the related art, when a user controls application software through an input device, an operation instruction of the user received by the input device needs to be forwarded to a server through a display device. Illustratively, fig. 1 is a schematic diagram of forwarding an operation instruction to a server through a presentation device. The input device 1-1 firstly sends an operation instruction 1-2 corresponding to the operation of a user to the display device 1-4 through the device interface 1-3, the operation system 1-41 of the display device 1-4 receives the operation instruction 1-2 from the device interface 1-3, and determines an input event 1-5 (information for performing function control on application software) corresponding to the operation instruction 1-2 and sends the input event 1-5 to the server 1-6 through the software client 1-42 running on the operation system 1-41.

After receiving an operation instruction of a user, the server controls the application software according to the operation instruction, and sends feedback content to the display device, for example, controls a cloud game running on the server, generates a game screen and the like, and then returns the game screen to the smart television and the like of the user.

For example, fig. 2 is a schematic diagram of a process for controlling a cloud game via an input device. The method comprises the steps that a user 2-1 operates 2-3 on an input device, for example, a game pad 2-2 to generate an operation instruction, the game pad 2-2 communicates with a display device, namely a smart television 2-4, so that the operation instruction of the user is sent to the smart television 2-4 provided with a cloud game client 2-41, the smart television 2-4 converts the operation instruction of the user into input information 2-5, and the input information is sent to a router 2-6. The router 2-6 sends the input information 2-5 to a server running a cloud game, namely the cloud game server 2-7, the cloud game server 2-7 generates a new game picture 2-8 aiming at the input information 2-5, and sends the game picture 2-8 to the smart television 2-4 through the router 2-6, so that a user can watch the game picture 2-9 on the smart television 2-4.

As can be seen from the above, in the related art, when the user controls the application software through the input device, the operation instruction generated by the input device needs to be forwarded at least once to reach the server. However, the operation instruction of the input device is forwarded to the server through the display device, which undoubtedly increases the time for the server to receive the operation instruction, so that the feedback content of the application software can be generated and issued only after a period of time for the user to operate, that is, there is a higher communication delay for the control of the application software.

Furthermore, higher communication delay exists for the control of the application software, which may cause the reduction of the user experience, and especially for cloud games such as combat games, which have higher requirements for real-time feedback, the operation experience of the user may be greatly affected, which may finally result in the loss of the user.

Embodiments of the present application provide an information interaction method, apparatus, device, computer-readable storage medium, and program product, which can reduce communication delay when controlling application software. The following describes exemplary applications of the server, the display device and the input device provided in the embodiments of the present application, where the display device provided in the embodiments of the present application may be implemented as various types of terminals such as a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device (e.g., a mobile phone, a portable game device), a smart television, and a smart watch, and the input device may be implemented as a game pad, a motion sensing device, a VR (Virtual Reality) device, and the like. Next, an exemplary application of the information interaction method will be explained.

Referring to fig. 3, fig. 3 is a schematic structural diagram of an information interaction system provided in an embodiment of the present application. In order to support an information interaction application, in the information interaction system 100, the presentation apparatus 400 and the input apparatus 500 are connected to the server 200 through the network 300, and the network 300 may be a wide area network or a local area network, or a combination of the two.

The server 200 is configured to issue an initial information stream for the presentation apparatus 400, wherein the initial information stream is used as a reference when an operation event is input; receiving a network data packet sent by the input device 500, and analyzing an operation event triggered by the initial information flow from the network data packet; generating an update information stream in response to the operational event; the update information stream is sent to presentation device 400 to cause presentation device 400 to display the update information stream.

The input device 500 is used for determining an operation event for the initial information stream being presented by the presentation device 400 in response to an input operation by a user; wherein, the initial information flow is used as a reference when inputting an operation event; packaging the operation event to obtain a network data packet; the network data packet is sent to the server 200, so that the server 200 generates an update information stream according to the operation event in the network data packet, and pushes the update information stream to the display apparatus 400.

The display device 400 is configured to receive an initial information stream sent by the server 200; wherein, the initial information flow is used as a reference when inputting an operation event; displaying an initial information stream on an information display interface in the graphical interface 400-1; receiving an update information stream sent by the server 200; the initial information flow of the information presentation interface is replaced with an updated information flow, wherein the updated information flow is generated by the server 200 in response to the operation event parsed from the network data packet sent by the input device 500.

In some embodiments, the server 200 may be an independent physical server, may also be a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a CDN, and a big data and artificial intelligence platform. The display device 400 may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart watch, a smart appliance, an in-vehicle device, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited in the embodiment of the present invention.

Referring to fig. 4, fig. 4 is a schematic structural diagram of the server in fig. 3 provided in an embodiment of the present application, where the server 200 shown in fig. 4 includes: at least one first processor 210, a first memory 250, at least one first network interface 220, and a first user interface 230. The various components in server 200 are coupled together by a first bus system 240. It is understood that the first bus system 240 is used to enable communications for connections between these components. The first bus system 240 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as first bus system 240 in fig. 4.

The first Processor 210 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc., wherein the general purpose Processor may be a microprocessor or any conventional Processor, etc.

The first user interface 230 includes one or more first output devices 231, including one or more speakers and/or one or more visual display screens, that enable presentation of media content. The first user interface 230 also includes one or more first input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

The first memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. The first memory 250 optionally includes one or more storage devices physically located remotely from the first processor 210.

The first memory 250 includes volatile memory or nonvolatile memory and may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM), and the volatile Memory may be a Random Access Memory (RAM). The first memory 250 described in embodiments herein is intended to comprise any suitable type of memory.

In some embodiments, the first memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplified below.

A first operating system 251 including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

a first network communication module 252 for communicating to other computing devices via one or more (wired or wireless) first network interfaces 220, an exemplary first network interface 220 comprising: bluetooth, wireless-compatibility authentication (Wi-Fi), and Universal Serial Bus (USB), etc.;

a first presentation module 253 to enable presentation of information (e.g., a user interface for operating peripherals and displaying content and information) via one or more first output devices 231 (e.g., a display screen, speakers, etc.) associated with the first user interface 230;

a first input processing module 254 for detecting one or more user inputs or interactions from one of the one or more first input devices 232 and translating the detected inputs or interactions.

In some embodiments, the information interaction device in the server provided by the embodiments of the present application may be implemented in software, and fig. 4 illustrates the information interaction device 255 stored in the first storage 250, which may be software in the form of programs and plug-ins, and includes the following software modules: a first sending module 2551, a first receiving module 2552 and an information generating module 2553, which are logical and thus can be arbitrarily combined or further split according to the implemented functions. The functions of the respective modules will be explained below.

In other embodiments, the information interaction Device 255 provided in this embodiment may be implemented in hardware, and as an example, the information interaction Device 255 provided in this embodiment may be a processor in the form of a hardware decoding processor, which is programmed to execute the server-side information interaction method provided in this embodiment, for example, the processor in the form of the hardware decoding processor may be one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components.

Referring to fig. 5, fig. 5 is a schematic structural diagram of the input device in fig. 3 according to an embodiment of the present disclosure, where the input device 500 shown in fig. 5 includes: at least one second processor 510, a second memory 550, at least one second network interface 520, and a second user interface 530. The various components in the input device 500 are coupled together by a second bus system 540. It will be appreciated that the second bus system 540 is used to enable communications among these components. The second bus system 540 includes a power bus, a control bus, and a status signal bus in addition to the data bus. But for clarity of illustration the various buses are labeled as the second bus system 540 in figure 5.

The second Processor 510 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor, or the like.

The second user interface 530 includes one or more second output devices 531 enabling presentation of media content, including one or more speakers and/or one or more visual displays. The second user interface 530 also includes one or more second input devices 532 including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

The second memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. The second memory 550 optionally includes one or more storage devices physically located remote from the second processor 510.

The second memory 550 may comprise either volatile memory or nonvolatile memory, and may also comprise both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM), and the volatile Memory may be a Random Access Memory (RAM). The second memory 550 described in embodiments herein is intended to comprise any suitable type of memory.

In some embodiments, the second memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplified below.

A second operating system 551 including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

a second network communication module 552 for communicating to other computing devices via one or more (wired or wireless) second network interfaces 520, the example second network interfaces 520 including: bluetooth, wireless-compatibility authentication (Wi-Fi), and Universal Serial Bus (USB), etc.;

a second rendering module 553 for enabling the rendering of information (e.g., user interfaces for operating peripherals and displaying content and information) via one or more second output devices 531 (e.g., display screens, speakers, etc.) associated with the second user interface 530;

a second input processing module 554 for detecting one or more user inputs or interactions from one of the one or more second input devices 532 and translating the detected inputs or interactions.

In some embodiments, the information interaction apparatus in the input device provided by the embodiments of the present application may be implemented in software, and fig. 5 illustrates an information interaction apparatus 555 stored in the second memory 550, which may be software in the form of programs and plug-ins, and includes the following software modules: the event determination module 5551, the data encapsulation module 5552, the second sending module 5553, the second receiving module 5554 and the network connection module 5555 are logical and thus may be arbitrarily combined or further split according to the implemented functions. The functions of the respective modules will be explained below.

In other embodiments, the information interaction Device 555 provided in this embodiment may be implemented in hardware, and as an example, the information interaction Device 555 provided in this embodiment may be a processor in the form of a hardware decoding processor, which is programmed to execute the information interaction method on the input Device side provided in this embodiment, for example, the processor in the form of the hardware decoding processor may be one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components.

Referring to fig. 6, fig. 6 is a schematic structural diagram of the display apparatus in fig. 3 according to an embodiment of the present disclosure, and the display apparatus 400 shown in fig. 6 includes: at least one third processor 410, a third memory 450, at least one third network interface 420, and a third user interface 430. The various components in the display device 400 are coupled together by a third bus system 440. It is understood that the third bus system 440 is used to enable connection communication between these components. The third bus system 440 includes a power bus, a control bus, and a status signal bus in addition to the data bus. For clarity of illustration, however, the various buses are labeled as the third bus system 440 in fig. 6.

The third Processor 410 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc., wherein the general purpose Processor may be a microprocessor or any conventional Processor, etc.

The third user interface 430 comprises one or more third output devices 431 comprising one or more speakers and/or one or more visual displays enabling presentation of the media content. The third user interface 430 also includes one or more third input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

The third memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. The third memory 450 optionally includes one or more storage devices physically located remote from the third processor 410.

The third memory 450 includes either volatile memory or nonvolatile memory, and may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM), and the volatile Memory may be a Random Access Memory (RAM). The third memory 450 described in embodiments herein is intended to comprise any suitable type of memory.

In some embodiments, third memory 450 is capable of storing data, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below, to support various operations.

A third operating system 451 including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

a third network communication module 452 for communicating to other computing devices via one or more (wired or wireless) third network interfaces 420, the exemplary third network interface 420 comprising: bluetooth, wireless-compatibility authentication (Wi-Fi), and Universal Serial Bus (USB), etc.;

a first rendering module 453 for enabling the rendering of information (e.g., user interfaces for operating peripherals and displaying content and information) via one or more third output devices 431 (e.g., display screens, speakers, etc.) associated with the third user interface 430;

a third input processing module 454 for detecting one or more user inputs or interactions from one of the one or more third input devices 432 and translating the detected inputs or interactions.

In some embodiments, the information interaction apparatus in the display device provided by the embodiment of the present application may be implemented in a software manner, and fig. 6 illustrates the information interaction apparatus 455 stored in the third memory 450, which may be software in the form of programs and plug-ins, and includes the following software modules: a third receiving module 4551, an information presentation module 4552, a third sending module 4553 and an information acquisition module 4554, which are logical and thus may be arbitrarily combined or further divided according to the functions implemented. The functions of the respective modules will be explained below.

In other embodiments, the information interaction Device 455 provided in this embodiment may be implemented in hardware, and as an example, the information interaction Device 455 provided in this embodiment may be a processor in the form of a hardware decoding processor, which is programmed to execute the information interaction method on the display Device side provided in this embodiment, for example, the processor in the form of the hardware decoding processor may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components.

In some embodiments, the presentation device, the input device, or the server may implement the information interaction method provided by the embodiments of the present application by running a computer program. For example, the computer program may be a native program or a software module in an operating system; can be a local (Native) Application program (APP), i.e. a program that needs to be installed in an operating system to run; or may be an applet, i.e. a program that can be run only by downloading it to the browser environment; but also an applet that can be embedded into any APP. In general, the computer programs described above may be any form of application, module or plug-in.

The embodiment of the application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, vehicle-mounted and the like. The information interaction method provided by the embodiment of the present application will be described below with reference to exemplary applications and implementations of the presentation device, the input device, and the server provided by the embodiment of the present application.

Referring to fig. 7, fig. 7 is a first flowchart of an information interaction method provided in the embodiment of the present application, and will be described with reference to the steps shown in fig. 7.

S101, the server issues an initial information stream for the display equipment.

The embodiment of the application is realized in a scene that application software running on a server is controlled through an input device and a corresponding control result is displayed by a display device, for example, a character in a game running on the server is controlled through the input device and the motion process of the character is displayed by the display device, or decoration design software running on the server is subjected to contour drawing through the input device and a rendered drawing result is displayed by the display device.

In the embodiment of the application, no matter whether the user inputs the operation event aiming at the application software running on the server through the input device or not, the server issues the initial information stream aiming at the display device, and the display device receives the initial information stream sent by the server, so that the follow-up user can refer to the initial information stream to determine the operation to be performed by the user. That is, the initial information stream is used as a reference when an operation event is input.

It will be appreciated that the initial information stream may be the information stream generated by the server in response to the start of the application software, for example, the initial interface at the start of the finishing software, or a virtual scene after the start of the game. The virtual scene after the game is started may be a virtual scene saved when the game is last exited, or may be a completely new virtual scene unrelated to the last game, and the present application is not limited herein.

The initial information stream may also be an information stream generated by the server according to a historical operation event triggered by the user in the previous round, for example, the user triggers a virtual object in the game to run in the virtual scene before the current time, and then the initial information stream is a picture stream of the virtual scene when the virtual object runs; when the user changes the color of the furniture designed in the decoration design software before the current time, the initial information flow is the picture of the furniture after the color is changed.

It should be noted that the initial information stream may include a static picture, for example, a main interface of the decoration software, a dynamic video stream, for example, an animation of a virtual scene, and an audio stream, for example, a sound effect of a game, and the like, and the application is not limited herein.

S102, the display device displays the initial information flow on the information display interface.

It will be appreciated that the information presentation interface is displayed in a graphical interface of the presentation device. The information display interface is triggered to be displayed after the display device triggers the application software to start.

S103, the input device responds to the input operation and determines an operation event aiming at the initial information flow displayed by the display device.

The presentation device displays the initial information stream, meaning that the application runs in the server. At this time, the user may refer to the initial information stream being displayed, operate on the input device to control the application software, and at this time, the input device may determine what operation the user needs to perform with respect to the initial information stream in response to the input operation of the user, so as to obtain an operation event. That is, the operational event embodies the user's control requirements for the initial information flow.

It is understood that in some embodiments, a touch screen may be disposed on the input device to receive an input operation of the user, in which case, the input operation may be a click, a double click, a long press, a slide, and the like on the input device by the user. In other embodiments, the input device may be provided with a physical joystick and a physical key to receive an input operation of a user, where the input operation may be an operation of dragging the joystick, pressing the physical key, or the like.

It should be noted that, when the input device receives an input operation, it actually receives an electronic signal triggered by the input operation of the user, and electronic signals of different types and durations may represent different operation events, for example, a short electric pulse may represent that a virtual object jumps once, two consecutive electric pulses may represent that the virtual object jumps continuously, and the input device may determine the operation event for the initial information stream according to the received electronic signal.

And S104, the input equipment packages the operation event to obtain a network data packet.

After the input device determines the operation event, the operation event can be used as service data to be encapsulated, and then protocol encapsulation is performed on the service data to be encapsulated, that is, a protocol header is added to the service data to be encapsulated, so that a network data packet capable of being transmitted on a network is obtained.

It can be understood that the input device may sequentially transmit the service data to be encapsulated to the lower layer according to the sequence of the TCP/IP protocol stack, the IPX/SPX protocol stack, or other network communication protocol stacks, so as to sequentially provide the protocol header corresponding to each layer of the service data, and add the information required during transmission through the protocol header to obtain the network data packet.

Taking TCP/IP protocol stack as an example, the input device may first transmit an operation event, that is, service data to be encapsulated to the application layer, add an FTP protocol header, then transmit the operation event to the transport layer, add a TCP protocol header, then transmit to the network layer, add an IP protocol header, finally transmit to the link layer, add a device driver, so as to obtain a network data packet.

Of course, when the input device transmits the service data to the lower layer, the input device may add a corresponding protocol header to the service data, and may also add a corresponding protocol trailer to the service data to obtain the network data packet.

In some embodiments, the input device may further encrypt the operation event, that is, the service data to be encapsulated, and then perform protocol encapsulation on the encrypted data to obtain a network data packet, so that security of the operation event in the transmission process may be ensured.

S105, the server receives the network data packet sent by the input device, and analyzes the operation event triggered by the initial information flow from the network data packet.

After the input equipment obtains the network data packet, the network data packet is sent to the server through the network, so that the server generates an update information stream according to the operation event in the network data packet, and pushes the update information stream to the display equipment. When the server receives the network data packet sent by the input device, the server decapsulates the network data packet to analyze an operation event triggered by the user for the initial information flow from the network data packet, so as to control the application software according to the operation event in the future and generate an update information flow for the operation event.

It can be understood that, when the server decapsulates the network data packet, it actually removes the contents of the protocol header and the protocol trailer in the network data packet, and only extracts the operation event. When the input device encapsulates the encrypted service data, namely the encrypted operation event, the data obtained after the server decapsulates is also the encrypted service data, and at this time, the server continues to decrypt the encrypted service data to obtain the operation event.

In some embodiments, the input device may be connected to a local area network where the display device is located, that is, the input device and the display device are connected to the same router, and then the network data packet is sent to the server through the router. In other embodiments, the input device may also directly access the mobile network and then send the network data packet to the server through the mobile network. That is, in the present application, the input device and the presentation device can both transmit the network packet to the server when they access the same network or different networks.

It should be noted that, in the present application, the network data packet is directly sent to the server by the input device, and the display device is not required to forward the network data packet from the input device, so that the time required by the display device to receive and process the network data packet is inevitably reduced, and the network data packet can be sent to the server more quickly.

It will be appreciated that the input device may query the presentation device to obtain a route for the server to send the network packet, or may obtain a route for the server from the router to send the network packet.

S106, the server responds to the operation event and generates an updating information flow.

The server can determine what kind of control the application software needs to be performed by the user according to the operation event, perform corresponding control on the application software, and render the feedback of the application software into an update information stream so as to inform the user that the function control of the application software is realized by using the update information stream.

For example, in some embodiments, when the operation event is to control a virtual scene in the game, the server updates the virtual scene according to the operation event, so as to obtain an updated information stream based on the updated virtual scene. In other embodiments, when the operation event is to control a three-dimensional model of furniture in the decoration software, the server updates the three-dimensional model according to the operation event, so as to obtain an updated information stream based on the updated three-dimensional model.

S107, the server sends the update information stream to the display device so that the display device displays the update information stream.

After the server generates the update information stream, the server returns the update information stream to the display device through the network, and the display device receives the update content stream sent by the server so as to be displayed subsequently.

And S108, replacing the initial information flow of the information display interface with the updated information flow by the display equipment.

The display device replaces the initial information flow displayed on the information display interface by using the updated information flow, namely, the display device only displays the updated information flow in the information display interface and does not continuously display the initial information flow any more, so that a user can clearly determine a control result of the display device for the application software. Wherein the updated information flow is generated by the server in response to the operation event parsed from the network data packet sent by the input device.

Compared with the prior art, in the embodiment of the application, the input device directly sends the network data packet containing the operation event to the server by forwarding the operation instruction to the server through the display device, so that the time required for the display device to receive and process the network data packet can be reduced, the network data packet can be quickly sent to the server, the server quickly responds based on the operation event in the network data packet, generates the update information stream, and directly sends the update information stream to the display device for displaying. Therefore, the feedback content of the application software can be generated and issued in a short time after the user operates the application software, and the communication delay for controlling the application software is effectively reduced. Furthermore, communication time delay is reduced when the application software is controlled, the use experience of a user can be protected, and further the loss of the user is reduced.

Referring to fig. 8, fig. 8 is a schematic flowchart diagram of a second information interaction method provided in the embodiment of the present application. In some embodiments of the present application, the operational events include: the interaction event for interacting with the virtual object in the virtual scene, and generating the update information stream in response to the operation event, that is, the specific implementation process of S106, may include: S1061-S1062, as follows:

s1061, the server responds to the interaction event and updates the virtual scene to obtain an updated virtual scene.

When a user interacts with a virtual object in a virtual scene through an interaction event, a picture of the virtual scene changes, and therefore the server can respond to the interaction event and update the virtual scene. Wherein the updated virtual scene reflects the result of the interaction with the virtual object.

It is understood that the virtual object may be an object manipulated by the user in the virtual scene, for example, a game character of the user, etc.; the virtual object may also be an object that is not related to the user and is set in the virtual scene by the server, for example, an NPC, an object in the virtual scene, or the like; the virtual object may also be an object controlled by another user in the virtual scene, for example, a game character of another user, and the application is not limited herein.

S1062, rendering the updated virtual scene by the server to obtain an updated information stream.

After the server obtains the updated virtual scene, the server performs picture rendering on the updated virtual scene to generate a video stream of the updated virtual scene, and simultaneously generates an audio stream corresponding to the updated virtual scene, and combines the video stream and the audio stream to obtain an updated information stream.

It is understood that the video stream of the updated virtual scene describes the scene changes of the virtual scene, such as the position change, posture change, environment change, etc. of the virtual object; the audio stream describes the audio changes of the updated virtual scene, such as the transformation of the speaking content of the virtual object, the change of the sound in the environment, and the like. The updated information stream obtained by the video stream and the audio stream of the updated virtual scene is more vivid and attractive.

In the embodiment of the application, the server can generate and send the update information stream according to the interaction event with the virtual object in the virtual scene, so that the user can control the role in the game through the server.

In some embodiments of the present application, the interaction events include: one or more of an event that controls an action of a virtual object in the virtual scene, an event that controls an appearance of a virtual object in the virtual scene, and an event that controls an exchange of a virtual object in the virtual scene.

Wherein the action of the virtual object may include: walking, running, jumping, swimming, flying, riding, fighting movements, skill movements, etc.; the appearance of the virtual object may include: height, face, hair style, apparel style and color, accessories style and color, and weaponry style and color of the virtual object, among others; the communication of the virtual objects may include: communication with the NPC, task conversation, chat with other user-controlled avatars, etc., and the application is not limited thereto.

In the embodiment of the application, the interactive events comprise various different events, so that a user can perform various different controls on the virtual objects in the virtual scene according to the input equipment, and the user experience can be enriched.

Based on fig. 7, referring to fig. 9, fig. 9 is a third schematic flowchart of an information interaction method provided in the embodiment of the present application. In some embodiments of the present application, before the server issues the initial information stream for the display device, that is, before S101, the method may further include: S109-S112, as follows:

and S109, the display equipment sends a scene loading instruction to the server.

And S110, the server responds to the scene loading instruction and loads the virtual scene.

Before the display device receives the initial information stream of the server, the display device sends a scene loading instruction to the server to notify the server to load the virtual scene through the scene loading instruction, that is, the scene loading instruction is used for triggering the virtual scene loading. And the server receives a scene loading instruction sent by the display equipment.

Further, in the embodiment of the present application, the scene loading instruction is substantially an instruction used by the display terminal to notify the server of game start, and when receiving the instruction, the server determines that the user has a game starting demand to start loading the virtual scene.

And S111, rendering the virtual scene by the server to obtain an initial information stream.

The server renders the loaded virtual scene into an audio video stream, and then an initial information stream is obtained, so that the initial content stream comprises the picture and the sound effect of the loaded virtual scene, and the virtual scene is displayed in two dimensions of image and sound aiming at a user.

And S112, the display device sends the first routing information of the server to the input device.

In addition to sending the scene loading instruction to the server, the display device needs to notify the input device of the first routing information of the server itself, so that the input device can establish communication with the server according to the first routing information. The input device receives first routing information of the server sent by the display device.

In this case, the process of sending the network packet to the server may become: and sending the network data packet to a server according to the first routing information.

It will be appreciated that the presentation device may send the first routing information of the server to the input device via a wireless connection with the input device, for example, to the same router, or a bluetooth connection. The display device may also send the first routing information of the server to the input device through a wired connection with the input device, for example, a data line connection, which is not limited herein.

It should be noted that, in some embodiments, the presentation device may send the first routing information to the input device while sending the scene loading instruction to the server, that is, S109 and S112 may be executed synchronously; in other embodiments, the display device may further send the first routing information to the input device, and then send the scene loading instruction to the server, that is, perform S112 first, and then perform S109; in other embodiments, the display device may further send the first routing information to the input device when it is ensured that the server receives the scene loading instruction, for example, when a scene loading start prompt returned by the server is received, so that S109 is executed first, and then S112 is executed, which is not limited herein.

In the embodiment of the application, the display device can trigger the server to generate the initial information stream by sending the scene loading instruction to the server, start the game process, and transmit the first routing information of the server to the input device, so that the input device can directly communicate with the server according to the first routing information, and the time delay of sending the operation event to the server by the input device is reduced.

In some embodiments of the present application, before the presentation device sends the scene loading instruction to the server, that is, before S109, the method may further include: S113-S117, as follows:

s113, the input device is connected with the display device through hardware, and sends an information acquisition request to the display device.

Because there may not be many keys arranged on the input device, the user is supported to input the name and password of the network on the input device to access the network, at this time, the input device needs to acquire the information of the local area network connected with the display device from the display device by means of the hardware connection with the display device, and connect the information to the local area network connected with the display device, so as to facilitate subsequent direct communication with the server. In this case, the input device sends an information acquisition request for acquiring network connection information to the display device through a hardware connection with the display device. The display device receives an information acquisition request sent by the input device.

It should be understood that the hardware connection may refer to connection via a USB interface, connection via a data line, etc., as long as the connection is not via the lan corresponding to the router.

S114, the display device responds to the information acquisition request, and acquires the network connection information and second routing information of the display device.

After receiving the information acquisition request, the display device acquires information of the local area network connected with the display device, namely network connection information. Meanwhile, the display device acquires the routing information of the display device, namely the second routing information, so that the routing information can be sent to the input device in the subsequent process.

It is understood that the network connection information at least includes a network identifier (ID, name, etc.) and a password of a local area network where the display device is located, and the input device may be connected to the local area network where the display device is connected through the name and the password of the network, that is, connected to the same router as the display device. The local area network may be a wired network or a wireless network.

And S115, the display device sends the network connection information and the second routing information to the input device.

The input device receives the network connection information and the second routing information sent by the display device aiming at the information acquisition request.

And S116, the input equipment accesses the local area network where the display equipment is located according to the network connection information.

After the input device obtains the network connection information, the name and the password of the local area network where the display device is located are analyzed from the network connection information, and then the display device is connected to the local area network.

And S117, the input device sends a prompt message of successful network access to the display device according to the second routing information.

After the input device is connected to the wireless network, the presentation device needs to be informed that it is currently available, i.e. can be used to communicate with the server. At this time, the input device generates a prompt message for prompting the display device that the display device has successfully accessed the network, and then sends the prompt message to the display device by means of the local area network through the second routing information of the display device. The display device receives a prompt message of successfully accessing the network, which is sent by the input device, responds to the prompt message, pops up a prompt interface, and displays operation prompt information on the prompt interface to prompt the user to operate by using the input device, namely, prompts the user that the input device is available at present and can be used for operating.

In the embodiment of the application, the input device can obtain the network connection information of the wireless network from the display device through hardware connection and access the wireless network in which the display device is located by means of the network connection information, so that the input device can also successfully access the wireless network and directly communicate with the server under the condition that a user does not manually distribute the network.

Next, an exemplary application of the embodiment of the present application in a practical application scenario will be described.

The embodiment of the application is realized in the scene that a user carries out a cloud game. When a user carries out a cloud game, the user needs to control the cloud game through an input peripheral (input device), and a game scene (virtual scene) of the cloud game is displayed on an intelligent television (display device). Fig. 10 is a hardware deployment structure diagram of a running environment of a cloud game provided in an embodiment of the present application. Referring to fig. 10, the cloud game runtime environment includes input peripherals, i.e., a gamepad 10-1, a smart tv 10-2, a cloud game server (server) 10-3, and a home router 10-4. Wherein, a wireless network card is configured on the game handle 10-1, and the game handle 10-1 is connected to the home router 10-4 through the wireless network card so as to communicate with the cloud game server 10-3 through the home router 10-4. The cloud game client is installed on the smart television 10-2, and a general user can start a cloud game by starting the cloud game client on the smart television 10-2. After the cloud game is started, the smart tv 10-2 and the cloud game server 10-3 communicate through the home router 10-4.

Fig. 11 is a schematic diagram of a flow of a cloud game runtime provided in an embodiment of the present application. Referring to fig. 11, a user 11-1 performs an operation 11-3 on an input peripheral device, i.e., a game pad 11-2, generates input information 11-4 (operation event), and transmits the input information 11-4 to a home router 11-5 to transmit the input information 11-4 to a cloud game server 11-6 through the home router 11-5. After receiving the input information 11-4, the cloud game server 11-6 generates a game screen 11-7 (update information stream) in response to the input information, transmits the game screen 11-7 to the home router 11-5, so that the game screen 11-7 is forwarded to the smart television 11-8 running with the cloud game client 11-81 through the home router 11-5, and a user views the game screen 11-9 on the smart television 11-8.

Therefore, in the embodiment of the application, the uplink data, namely the input information, is directly sent to the external network link through the home router and finally sent to the cloud game server, and the downlink data, namely the game picture, is only sent to the smart television for displaying, and is irrelevant to input. Therefore, the communication between the input peripheral and the cloud game server is directly carried out without using the intelligent television as a transfer, and the time delay between the input peripheral and the cloud game server can be reduced.

It should be noted that the input peripheral device in the embodiment of the present application needs to support wireless network connection, can receive information from the smart television to perform network distribution, that is, directly connect to the home router, and can encapsulate input information triggered by a user into a network packet (network data packet) and send the network packet (network data packet) to the IP (first routing information) of the cloud game server.

Next, the operation process of the cloud game will be described with reference to the process of establishing communication between the input peripheral and the cloud game server.

Fig. 12 is a schematic diagram of a process for establishing communication between an input peripheral and a cloud game server according to an embodiment of the present application. Referring to fig. 12, the process includes:

s201, the input peripheral is connected to the home router to access a wireless network (access a local area network where the display equipment is located).

S202, the input peripheral informs the cloud game client running on the smart television, and the cloud game client and the input peripheral establish communication.

After the input peripheral is accessed to the home router, the smart television needs to be informed of successful access (a prompt message of successful network access is sent to the display device) through the local area network so as to receive information from the smart television.

S203, the smart television notifies the cloud game server to start the cloud game (send a scene loading instruction to the server), and transmits the IP of the cloud game server to the input peripheral (send the first routing information of the server to the input device).

The intelligent television can send the IP of the cloud game server to the input peripheral through a wireless network of the home router.

And S204, the input peripheral equipment directly communicates with the cloud game server through the network according to the received IP.

Thus, the cloud game can be executed.

Further, fig. 13 is a schematic diagram of a process for accessing a wireless network by an input peripheral according to an embodiment of the present application. Referring to fig. 13, a user may operate the smart tv 13-a installed with the cloud game client 13-a to connect the smart tv 13-a to a wireless network and record an ID and a password 13-1 (network connection information) of the wireless network; the input peripheral 13-B configured with the wireless network card 13-B reads the ID and the password of the wireless network from the smart television through a data line (hardware connection) or Bluetooth and records the local area network connection information 13-2 of the smart television (receives the network connection information and the second routing information sent by the display device aiming at the information acquisition request); the input peripheral 13-B establishes a connection 13-3 with the home router using the ID and password of the wireless network, thereby connecting to the home router 13-C.

According to the process shown in fig. 13, the input peripheral device can be connected to the wireless network and communicate directly with the cloud game server based thereon. Meanwhile, after the input peripheral equipment is connected to the wireless network, the direct connection with the hardware of the intelligent television can be disconnected, and the input peripheral equipment and the intelligent television are in direct communication in the local area network through the home router. The subsequent communication between the input peripheral and other devices is completed through the local area network.

After the input peripheral is connected into the local area network, the smart television needs to be informed of successful access through the local area network so as to receive information from the smart television. In the process shown in fig. 13, the input peripheral device records the connection information of the local area network of the smart television, and can directly send a notification to the smart television (send a prompt message for successful network access to the display device according to the second routing information) to notify that the smart television successfully accesses the network, and can be used as an input device of the cloud game.

For example, fig. 14 is a schematic diagram of a process in which an input peripheral sends a notification of successful access to a smart television according to an embodiment of the present application. The input peripheral 14-A configured with the wireless network card 14-a sends an access notification 14-1 to the smart television in the local area network through the home router 14-B, and the smart television 14-C provided with the cloud game client 14-C receives the notification from the input peripheral through the home router 14-B, confirms that the input peripheral is available, and records the local area network IP 14-2. Thus, the input peripheral completes the notification of the intelligent television.

Based on the processes shown in fig. 13 and 14, the input peripheral can acquire the IP address of the cloud game server. At this time, the smart television informs the cloud game server to start the game, and the input peripheral needs to send input information to the cloud game server to control the cloud game.

Referring to fig. 15, fig. 15 is a schematic diagram of a process for starting a cloud game according to an embodiment of the present application. The intelligent television 15-A provided with the cloud game client 15-a informs the cloud game server to start a cloud game through the home router 15-B, and sends the IP 15-1 of the cloud game server to the input peripheral through the local area network; the cloud game server 15-C runs the cloud game and waits for the arrival of the input information 15-2; the input peripheral 15-D configured with the wireless network card 15-D receives the IP of the cloud game server and sends input information 15-3 to the cloud game server according to the IP; the cloud game server 15-C receives input information from the network, and returns a game screen 15-4 in response. Therefore, the cloud game is started, and the user can control the cloud game.

After receiving the notification of starting the cloud game sent by the smart television, the cloud game server continuously returns the game picture to the smart television regardless of whether input information sent by the input peripheral is received. However, the game screen at this time is not affected by the user operation (i.e., the initial information flow), and the user can control the cloud game based on the game screen at this time to generate a new game screen (update information flow).

Through the mode, when the input peripheral sends the input information of the user to the cloud game server, the intelligent television is not required to forward the input information, namely, the input peripheral and the cloud game server realize direct communication. Therefore, the time required for transmitting the input information in the cloud game to the cloud game server can be reduced, so that the cloud game server can obtain the input information more quickly and make a response more quickly, a corresponding game picture is generated, and the communication time delay of the user in controlling the cloud game is reduced.

Continuing with the exemplary structure of the information interaction device 255 provided by the embodiments of the present application as software modules, in some embodiments, as shown in fig. 4, the software modules stored in the information interaction device 255 of the first memory 250 may include:

a first sending module 2551, configured to issue an initial information stream for the display device; wherein the initial information flow is used as a reference when an operation event is input;

a first receiving module 2552, configured to receive a network data packet sent by an input device, and parse the operation event triggered for the initial information flow from the network data packet;

an information generating module 2553, configured to generate an update information stream in response to the operation event;

the first sending module 2551 is further configured to send the update information stream to the display device, so that the display device displays the update information stream.

In some embodiments of the present application, the operational event comprises: an interaction event for interacting with a virtual object in a virtual scene; the information generating module 2553 is further configured to respond to the interaction event, update the virtual scene, and obtain an updated virtual scene; rendering the updated virtual scene to obtain the updated information stream.

In some embodiments of the present application, the first receiving module 2552 is further configured to receive a scene loading instruction sent by the display apparatus; the scene loading instruction is used for triggering loading of the virtual scene;

the information generating module 2553 is further configured to load the virtual scene in response to the scene loading instruction; rendering the virtual scene to obtain the initial information flow; the initial information stream includes the loaded pictures and sound effects of the virtual scene.

Continuing with the exemplary structure of the information interaction device 555 provided by the embodiments of the present application as a software module, in some embodiments, as shown in fig. 5, the software module stored in the information interaction device 555 in the second memory 550 may include:

an event determination module 5551, configured to determine, in response to an input operation, an operation event for an initial information stream being presented by the presentation apparatus; wherein the initial information flow is used as a reference when an operation event is input;

a data encapsulation module 5552, configured to encapsulate the operation event to obtain a network data packet;

a second sending module 5553, configured to send the network data packet to a server, so that the server generates an update information stream according to an operation event in the network data packet, and pushes the update information stream to the display device.

In some embodiments of the present application, the information interaction device 555 further includes: the second receiving module 5554; the second receiving module 5554 is configured to receive the first routing information of the server sent by the display device;

the second sending module 5553 is further configured to send the network data packet to the server according to the first routing information.

In some embodiments of the present application, the information interaction device 555 further includes: a network connection module 5555;

the second sending module 5553 is further configured to send an information obtaining request to the display apparatus through a hardware connection with the display apparatus;

the second receiving module 5554 is further configured to receive network connection information and second routing information sent by the display apparatus for the information acquisition request; the network connection information at least comprises a network identifier and a password of a local area network where the display equipment is located;

the network connection module 5555 is configured to access the local area network where the display device is located according to the network connection information;

the second sending module 5553 is further configured to send, according to the second routing information, a prompt message that the network is successfully accessed to the display device.

Continuing with the exemplary structure of the information interaction device 455 provided by the embodiments of the present application as software modules, in some embodiments, as shown in fig. 6, the software modules stored in the information interaction device 455 of the third memory 450 may include:

a third receiving module 4551, configured to receive an initial information stream sent by the server; wherein the initial information flow is used as a reference when an operation event is input;

an information presentation module 4552, configured to present the initial information stream on an information presentation interface;

the third receiving module 4551 is further configured to receive an update information stream sent by the server;

the information presentation module 4552 is further configured to replace the initial information flow of the information presentation interface with the updated information flow, where the updated information flow is generated by the server in response to an operation event parsed from a network data packet sent by an input device.

In some embodiments of the present application, the information interaction device 455 further includes: a third transmitting module 4553; the third sending module 4553 is configured to send a scene loading instruction to the server; the scene loading instruction is used for triggering loading of a virtual scene; and sending the first routing information of the server to the input equipment.

In some embodiments of the present application, the information interaction device 455 further includes: an information acquisition module 4554;

the third receiving module 4551 is further configured to receive an information acquisition request sent by an input device;

the information obtaining module 4554 is configured to, in response to the information obtaining request, obtain network connection information and second routing information of the display device; the network connection information at least comprises a network identifier and a password of a local area network where the display equipment is located;

the third sending module 4553 is further configured to send the network connection information and the second routing information to the input device;

the third receiving module 4551 is further configured to receive a prompt message of successful network access sent by the input device;

the information presentation module 4552 is further configured to, in response to the prompt message, present operation prompt information on a prompt interface to prompt an operation performed by using the input device.

Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the information interaction method described in the embodiment of the present application.

The embodiment of the application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored, when being executed by a first processor, the executable instructions will cause the first processor to execute the information interaction method on the server side provided by the embodiment of the application, when being executed by a second processor, the executable instructions will cause the second processor to execute the information interaction method on the input device side provided by the embodiment of the application, and when being executed by a third processor, the executable instructions will cause the third processor to execute the information interaction method on the display device side provided by the embodiment of the application.

In some embodiments, the computer-readable storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions may correspond, but do not necessarily have to correspond, to files in a file system, and may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on one computing device (server, input device, and presentation device), or on multiple computing devices located at one site, or distributed across multiple sites and interconnected by a communication network.

In summary, according to the embodiment of the present application, the input device directly sends the network data packet including the operation event to the server, so that the time required for the display device to receive and process the network data packet can be reduced, the network data packet can be quickly sent to the server, the server quickly responds based on the operation event in the network data packet, generates the update information stream, and directly sends the update information stream to the display device for display. Therefore, the feedback content of the application software can be generated and issued in a short time after the user operates the application software, and the communication delay for controlling the application software is effectively reduced. Furthermore, communication time delay is reduced when the application software is controlled, the use experience of a user can be protected, and further the loss of the user is reduced.

The above description is only an example of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An information interaction method, characterized in that the information interaction method comprises:

generating an update information stream in response to the operational event;

2. The method of claim 1, wherein the operational event comprises: an interaction event for interacting with a virtual object in a virtual scene; generating, in response to the operational event, an update information stream, comprising:

responding to the interaction event, and updating the virtual scene to obtain an updated virtual scene;

rendering the updated virtual scene to obtain the updated information stream.

3. The method of claim 2, wherein the interaction event comprises: one or more of an event that controls an action of a virtual object in the virtual scene, an event that controls an appearance of the virtual object in the virtual scene, and an event that controls an exchange of the virtual object in the virtual scene.

4. The method according to claim 2 or 3, wherein before issuing the initial information stream for the display device, the method further comprises:

receiving a scene loading instruction sent by the display equipment; the scene loading instruction is used for triggering loading of the virtual scene;

loading the virtual scene in response to the scene load instruction;

rendering the virtual scene to obtain the initial information flow; the initial information stream includes the loaded pictures and sound effects of the virtual scene.

5. An information interaction method, characterized in that the information interaction method comprises:

packaging the operation event to obtain a network data packet;

6. The method of claim 5, wherein prior to determining the operational event for the initial information stream being presented by the presentation device in response to the input operation, the method further comprises:

receiving first routing information of the server sent by the display equipment;

the sending the network data packet to a server includes:

and sending the network data packet to the server according to the first routing information.

7. The method according to claim 6, wherein before receiving the first routing information of the server sent by the display device, the method further comprises:

sending an information acquisition request to the display equipment through hardware connection with the display equipment;

receiving network connection information and second routing information sent by the display equipment aiming at the information acquisition request; the network connection information at least comprises a network identifier and a password of a local area network where the display equipment is located;

accessing the local area network where the display equipment is located according to the network connection information;

and sending a prompt message of successfully accessing the network to the display equipment according to the second routing information.

8. An information interaction method, characterized in that the information interaction method comprises:

displaying the initial information flow on an information display interface;

receiving an update information stream sent by the server;

9. The method of claim 8, wherein before the receiving the initial information stream sent by the server, the method further comprises:

sending a scene loading instruction to the server; the scene loading instruction is used for triggering loading of a virtual scene;

and sending the first routing information of the server to the input equipment.

10. The method of claim 9, wherein prior to sending the scene load instruction to the server, the method further comprises:

receiving an information acquisition request sent by input equipment;

responding to the information acquisition request, and acquiring network connection information and second routing information of the display equipment; the network connection information at least comprises a network identifier and a password of a local area network where the display equipment is located;

sending the network connection information and the second routing information to the input device;

and receiving a prompt message of successfully accessing the network, which is sent by the input equipment, and responding to the prompt message, and displaying operation prompt information on a prompt interface to prompt the input equipment to operate.

11. An information interaction apparatus, characterized in that the information interaction apparatus comprises:

12. An information interaction apparatus, characterized in that the information interaction apparatus comprises:

13. An information interaction apparatus, characterized in that the information interaction apparatus comprises:

14. A server, characterized in that the server comprises:

a first memory for storing executable instructions;

a first processor, configured to implement the information interaction method according to any one of claims 1 to 4 when executing the executable instructions stored in the first memory.

15. An input device, characterized in that the input device comprises:

a second memory for storing executable instructions;

a second processor, configured to implement the information interaction method according to any one of claims 5 to 7 when executing the executable instructions stored in the second memory.

16. A display apparatus, characterized in that the display apparatus comprises:

a third memory for storing executable instructions;

a third processor, configured to execute the executable instructions stored in the third memory, and implement the information interaction method according to any one of claims 8 to 10.

17. A computer-readable storage medium storing executable instructions, wherein the executable instructions when executed by a first processor implement the information interaction method of any one of claims 1 to 4, or when executed by a second processor implement the information interaction method of any one of claims 5 to 7, or when executed by a third processor implement the information interaction method of any one of claims 8 to 10.

18. A computer program product comprising a computer program or instructions, characterized in that the computer program or instructions, when executed by a first processor, implement the information interaction method of any one of claims 1 to 4, or when executed by a second processor, implement the information interaction method of any one of claims 5 to 7, or when executed by a third processor, implement the information interaction method of any one of claims 8 to 10.