WO2023001086A1 - 基于云应用的设备控制方法、装置、电子设备和可读介质 - Google Patents

基于云应用的设备控制方法、装置、电子设备和可读介质 Download PDF

Info

Publication number
WO2023001086A1
WO2023001086A1 PCT/CN2022/106151 CN2022106151W WO2023001086A1 WO 2023001086 A1 WO2023001086 A1 WO 2023001086A1 CN 2022106151 W CN2022106151 W CN 2022106151W WO 2023001086 A1 WO2023001086 A1 WO 2023001086A1
Authority
WO
WIPO (PCT)
Prior art keywords
cloud application
vibration feedback
scene
vibration
cloud
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/106151
Other languages
English (en)
French (fr)
Inventor
徐士立
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to JP2023561072A priority Critical patent/JP7670852B2/ja
Priority to KR1020237033698A priority patent/KR102939499B1/ko
Publication of WO2023001086A1 publication Critical patent/WO2023001086A1/zh
Priority to US18/239,665 priority patent/US12569756B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/358—Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/355—Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/25—Output arrangements for video game devices
    • A63F13/28—Output arrangements for video game devices responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light
    • A63F13/285—Generating tactile feedback signals via the game input device, e.g. force feedback
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/55—Controlling game characters or game objects based on the game progress
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/55—Controlling game characters or game objects based on the game progress
    • A63F13/57—Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/30—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device
    • A63F2300/302—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device specially adapted for receiving control signals not targeted to a display device or game input means, e.g. vibrating driver's seat, scent dispenser
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/40—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterised by details of platform network
    • A63F2300/402—Communication between platforms, i.e. physical link to protocol
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/50—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/50—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers
    • A63F2300/57—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers details of game services offered to the player
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Definitions

  • the present application relates to the field of computer technology, and in particular to a cloud application-based device control method, device, electronic device and readable medium.
  • Games are one of the most important forms of entertainment, especially for the current popular cloud games, which can provide users with an excellent entertainment experience without requiring users to have high-end hardware.
  • the application client on the server needs to render the application screen and scene into a video and send it to the user terminal, and play it on the user terminal to display the interface content .
  • the terminal device can only receive the rendered audio and video stream, which can provide limited interaction methods, and it is difficult to provide a rich and immersive experience, which affects the interaction between the cloud application and the user. Effect.
  • the present application provides a cloud application-based device control method, device, electronic equipment, and readable medium, so that in the cloud application scenario, the server can trigger vibration feedback that matches the application content, thereby improving The accuracy of vibration feedback improves user experience.
  • a device control method based on a cloud application executed by an electronic device, including:
  • a device control device based on a cloud application including:
  • a communication establishment module configured to establish a communication connection with the cloud application client on the terminal device, and the cloud application client is used to present the received cloud application video stream;
  • a scene identification module configured to perform scene identification on the cloud application corresponding to the cloud application client, and obtain a scene identification result, the scene identification result including media content associated with the identified cloud application scene;
  • a vibration feedback information generating module configured to generate vibration feedback information corresponding to the cloud application scenario according to the scene recognition result
  • a vibration feedback information sending module configured to send the video stream corresponding to the cloud application scene and the vibration feedback information to the cloud application client, so that the cloud application client presents the video stream of the cloud application scene
  • the vibration feedback information When, according to the vibration feedback information, control the physical device associated with the cloud application client to perform the vibration feedback operation.
  • a device control method based on a cloud application executed by an electronic device, the method comprising:
  • the vibration feedback information is generated by the application client on the server according to the cloud application scene;
  • a device control device based on a cloud application including:
  • a communication connection establishment module configured to establish a communication connection with the cloud application server
  • the vibration feedback information receiving module is used to receive the video stream and vibration feedback information corresponding to the cloud application scene sent by the cloud application server, and the vibration feedback information is generated by the application client on the server according to the cloud application scene;
  • a playback module configured to play the video stream corresponding to the cloud application scenario
  • the vibration feedback execution module is configured to control the physical device associated with the cloud application client to perform a vibration feedback operation according to the vibration feedback information.
  • an electronic device the electronic device includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions by executing the Such as the device control method based on cloud application in the above technical solution.
  • a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the cloud application-based device control method in the above technical solution is implemented.
  • a computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the device control method based on the cloud application provided in the above various optional implementation manners.
  • FIG. 1 is an example network architecture diagram of a cloud application-based device control system in an embodiment of the present application
  • FIG. 2 is a schematic diagram of a cloud application device control framework in an embodiment of the present application
  • Fig. 3 is a schematic diagram of the vibration special effect design process in this application.
  • FIG. 4 is a flow chart of a cloud application-based device control method provided in an embodiment of the present application.
  • Fig. 5 is the process of triggering vibration feedback in the embodiment of the present application.
  • Fig. 6 is the process of establishing a communication connection with the cloud application client in the embodiment of the present application.
  • FIG. 7 is a flow chart of a cloud application-based device control method provided in another embodiment of the present application.
  • Fig. 8A schematically shows a block diagram of a device control device based on a cloud application in an embodiment of the present application
  • FIG. 8B schematically shows a block diagram of a device control device based on cloud applications in another embodiment of the present application
  • FIG. 9 shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present application.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concepts of example embodiments to those skilled in the art.
  • cloud application refers to the embodiment of cloud computing technology at the application layer.
  • the working principle of “cloud application” is to change the traditional application (or software) "local installation, local computing” usage method into an “out-of-the-box” service, and connect and control remote server clusters through the Internet or a local area network to complete Business logic or computing tasks.
  • the main carrier of "cloud application” is Internet technology.
  • the cloud application client is presented in the form of thin client (Thin Client) or smart client (Smart Client), and its interface is the integration of technologies such as HTML5, Javascript or Flash.
  • Cloud applications can not only help users reduce IT costs, but also greatly improve work efficiency.
  • the device control system based on cloud applications can run in the network architecture shown in FIG. 1 .
  • the device control system based on cloud applications includes a cloud server 101 and various types of terminal devices 102-106.
  • the cloud server 101 can provide the control process of cloud applications for various terminal devices 102-106, that is, The control operation on the terminal device 102-106 side sends the operation command to the cloud server 101, and the cloud server 101 performs cloud application instance control, scene recognition, operation result calculation, audio and video rendering and other processing for the operation command, and sends the terminal device 102- Step 106 returns the information corresponding to the operation instruction, such as picture, music and so on.
  • FIG. 1 shows various terminal devices 102-106, and the terminal devices may be computer devices. In actual scenarios, more or fewer types of terminal devices may participate in the process of controlling cloud applications. , the specific quantity and type depend on the actual scene, and are not limited here.
  • one cloud server 101 is shown in FIG. 1 , but in actual scenarios, there may also be multiple cloud servers participating, especially in scenarios where multiple different cloud game It depends on the scene.
  • the cloud server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, Cloud servers for basic cloud computing services such as middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
  • the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.
  • the terminal and the cloud server can be directly or indirectly connected through wired or wireless communication, and the terminal and the cloud server can be connected to form a blockchain network, which is not limited in this application.
  • cloud gaming is an online gaming technology based on cloud computing technology, also known as gaming on demand.
  • Cloud gaming technology enables light-end devices (such as thin clients) with relatively limited graphics processing and data computing capabilities to run high-quality games.
  • cloud game scenarios such as cloud game platforms, etc., including cloud servers and game terminal devices, wherein the cloud server contains a device system or program for cloud game control functions, and the game terminal is used for Various operations of the user are controlled during the user's game.
  • cloud server contains a device system or program for cloud game control functions
  • game terminal is used for Various operations of the user are controlled during the user's game.
  • the game does not run on the player's game terminal device, but runs on the cloud server, and the cloud server renders the game scene into an audio and video stream, which is transmitted to the player's game terminal device through the network.
  • the player's game terminal device does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities, and the ability to obtain player input instructions and send them to the cloud server.
  • the device control method based on the cloud game can run on a personal mobile terminal, a cloud server, or a third-party device to provide cloud game control.
  • the specific cloud game device control system can run in the above-mentioned devices in the form of a program, or can run as a system component in the above-mentioned devices, and can also be used as a cloud service program.
  • the specific operation mode depends on the actual scene. , is not limited here.
  • FIG. 2 is a schematic diagram of a cloud application device control framework in an embodiment of the present application, including an application client 210 on a server, a cloud application server 220 and a cloud application client 230 on a terminal device.
  • the application client 210 sends an instruction to the cloud application server 220 through the terminal identification module 211, and the instruction finally reaches the cloud application client 230, so as to query the terminal device (such as an intelligent terminal) where the cloud application client 230 is currently located or communicate with the cloud application client 230.
  • the terminal device such as an intelligent terminal
  • the physical device associated with the application client 230 supports vibration feedback, and the format and content of the supported vibration feedback.
  • the terminal control module 213 establishes communication with the cloud application client 230 according to the information provided by the terminal identification module 211 , and the communication is relayed by the cloud application server 220 to issue vibration feedback information through the established communication.
  • the vibration feedback control module 212 of the application client 210 triggers corresponding vibration feedback according to different application scenarios, obtains the actual file of the vibration feedback information, and sends the file to the cloud application server 220 through the terminal control module 213.
  • the cloud application server 220 relays, and finally delivers to the cloud application client 230 .
  • the cloud application server 220 establishes a link with the cloud application client 230 through the terminal communication module 221 , obtains support information of vibration feedback from the cloud application client 230 , and sends a vibration feedback control protocol to the cloud application client 230 .
  • the vibration protocol packaging module 222 is used to receive the vibration feedback information delivered by the application client 210 , and package the vibration feedback information according to a defined protocol, and transmit the vibration feedback information to the cloud application client 230 through the terminal communication module 221 .
  • the cloud application client 230 establishes a communication connection with the cloud application server 220 according to the request of the cloud application server 220 through the communication module 231 , and receives information such as hardware query instructions and vibration feedback instructions issued by the cloud application server 220 .
  • the vibration feedback analysis module 232 is used for analyzing according to the defined protocol after receiving the vibration feedback instruction from the communication module 231, and calling the hardware control module 233.
  • the hardware control module 233 is used to receive instructions from the vibration feedback analysis module 232, and call the corresponding interface of the terminal device for actual hardware driving, so as to realize vibration feedback.
  • the application client 210 refers to the game client
  • the cloud application server 220 refers to the cloud game server
  • the cloud application client 230 refers to the cloud game client.
  • FIG. 3 is a schematic diagram of the vibration special effect design process in the embodiment of the present application, including the application client 310 on the server, the cloud application server 320 and the cloud application client on the terminal device. 330 interaction between.
  • the effect design 311 means that the server or the application client 310 on the server designs an exclusive vibration feedback effect for a specific scene.
  • Scene recognition 312 refers to that during the running of an application (such as a game), the server or application client 310 recognizes a corresponding scene and matches a related special effect description file.
  • an application such as a game
  • the server or application client 310 recognizes a corresponding scene and matches a related special effect description file.
  • different firearms used by the user can trigger different vibration feedback; when the user's virtual character is hit, the vibration of different intensities will be triggered according to the amount of blood loss, so that the user can directly perceive the blood loss through vibration Quantity; when the user encounters an enemy, a high-frequency short vibration is triggered to remind the user that danger is coming.
  • the vibration trigger 313 refers to that the application client 310 on the server sends a vibration request to request to call the vibration trigger interface of the physical device.
  • the cloud application is rendered on the cloud application server 320 , the request cannot be directly delivered to the physical device, so it needs to be transferred via the cloud application server 320 .
  • the transparent transmission terminal 321 means that the cloud application server 320 transparently transmits the vibration request to the cloud application client 330 through a communication connection after receiving the vibration request from the application client 310 .
  • the system trigger 331 refers to that the cloud application client 330 calls the interface of the physical device to trigger the vibration of the hardware on the terminal device after receiving the vibration request transparently transmitted by the cloud application server 320 .
  • the method provided by this application can be written as a program, as a processing logic in the hardware system, or as a cloud application control device, and the above processing logic can be realized in an integrated or external manner .
  • FIG. 4 is a flow chart of a cloud application-based device control method provided by an embodiment of this application. Execution, for example, by the application client 210 on the server in Figure 2, includes the following steps:
  • Step S301 establishing a communication connection with the cloud application client on the terminal device, and the cloud application client is used to present the received cloud application video stream.
  • the application client on the server establishes a communication connection with the cloud application client on the terminal device.
  • the application client is a specific application running on the server, and the cloud application client is an application on the terminal device for presenting the received cloud application video stream.
  • the cloud application client is a cloud game client.
  • the cloud game client can be a browser, a player, or a dedicated client on a mobile phone, which is mainly used to play the game screen provided by the server, while the game client is a racing car running on the server.
  • the practical application of the game is a racing game client.
  • the communication connection in this embodiment is mainly used to transmit information related to vibration feedback.
  • the communication connection can be a dedicated communication channel, or it can also be used for video stream communication and control information communication between the application client and the cloud application client. .
  • step S302 scene recognition is performed on the cloud application corresponding to the cloud application client, and a scene recognition result is obtained, and the scene recognition result includes media content associated with the recognized cloud application scene.
  • the cloud application is actually run by the application client on the server.
  • the application client will identify the scene where the cloud application is located according to the execution status of the cloud application and the user's operation.
  • game scenes usually include game progress, game screens, game data, etc. Changes in various data during the game may lead to changes in game scenes, for example, changes in the environment of virtual characters in the game, life Attributes such as values change, virtual characters encounter various events or interact with the environment, etc.
  • the application client can identify a specific cloud application scenario according to the data content and data changes of the cloud application, so as to obtain the scene identification result.
  • the scene recognition result includes the media content associated with the recognized cloud application scene, which is used to indicate the performance of the virtual character of the cloud application in the next period of time. For example, in a cloud game, if the virtual character falls from a high place, the scene recognition result can indicate that, in the next period of time, the virtual character falls on the ground and the damage received due to the fall from a high place.
  • Step S303 generating vibration feedback information corresponding to the cloud application scene according to the scene recognition result.
  • the cloud application scene identified in the scene recognition result can determine the changes or events that will occur in the cloud application in the next period of time, so as to determine the vibration feedback that needs to be triggered in the cloud application scene, Further, vibration feedback information corresponding to the identified cloud application scene is generated.
  • the vibration feedback information includes the type of vibration feedback to be performed and parameters for performing vibration feedback, such as execution time.
  • the vibration feedback information may include one or more parameters corresponding to the vibration feedback.
  • the vibration feedback information may include parameters required for multiple vibration feedbacks. For example, if in a cloud game scene, the target character collides with an obstacle three times during its fall, and vibration feedback is required for each collision, the vibration feedback information will include vibration feedback corresponding to three groups of vibrations corresponding to the three collisions parameter.
  • Step S304 sending the video stream corresponding to the cloud application scene and the vibration feedback information to the cloud application client, so that when the cloud application client presents the video stream of the cloud application scene, according to the vibration feedback information, the control is associated with the cloud application client.
  • FIG. 5 shows the process of triggering vibration feedback in the embodiment of the present application, including the following steps:
  • Step 501 the application client sends the video stream and vibration feedback information corresponding to the cloud application scene to the cloud application client through the cloud application server.
  • the video stream of the cloud application scene is a video stream rendered according to the application screen to be displayed in the cloud application scene.
  • Step 502 after receiving the video stream, the cloud application client plays the video stream to the user.
  • the cloud application client displays video content to the user, such as the result of the user's operation.
  • Step 503 when playing the video stream, the cloud application client also controls the associated physical device to perform a vibration feedback operation according to the received vibration feedback information.
  • the physical device associated with the cloud application client may be an intelligent terminal where the cloud application client is located.
  • the cloud application is a racing game
  • the smart terminal where the cloud application client is located is a mobile phone.
  • the video stream of the cloud game scene sent by the game client on the server will show the picture of the vehicle colliding with the opponent.
  • the cloud game client will also control the mobile phone to vibrate for 1 second, so as to show the impact of the impact and improve the user experience and immersion.
  • Step 504 after the physical device completes the vibration feedback, it returns the vibration execution result to the cloud application client.
  • Step 505 the cloud application client generates a vibration feedback response message including the vibration execution result, and sends the vibration feedback response message to the application client through the cloud application server.
  • the scene recognition result is obtained through scene recognition, the vibration feedback information corresponding to the cloud application scene is generated, and the vibration feedback information is sent to the cloud application client through the communication connection with the cloud application client, so as to Control the associated physical device to perform vibration feedback operation.
  • the vibration driving signal is sent in real time, and its delay is consistent with the video stream of the cloud application, so that users can experience zero-delay vibration and improve The accuracy of the vibration feedback is improved, thereby improving the user experience.
  • step S301 establishing a communication connection with the cloud application client on the terminal device, may include the following steps:
  • the communication response message includes the type of vibration feedback supported by the physical device and the protocol version of vibration feedback;
  • the generating vibration feedback information corresponding to the cloud application scene according to the scene recognition result includes:
  • the vibration feedback information is generated according to the scene recognition result, the vibration feedback type, and the protocol version of the vibration feedback.
  • the application client on the server sends a communication establishment request to the cloud application client.
  • the communication establishment request may be sent based on the established audio/video communication link or control communication link between the application client and the cloud application client, and may be performed at any time before the cloud application is officially started. For example, when the cloud application client logs in to an account or establishes other communication connections at the cloud application client, a communication connection for vibration feedback is established together.
  • FIG. 6 shows the process of establishing a communication connection with the cloud application client in the embodiment of the present application, including the following steps:
  • Step 601 the application client sends a communication establishment request to the cloud application client through the cloud application server according to the list of cloud application clients that have logged in to the server.
  • Step 602 upon receiving the communication establishment request, the cloud application client establishes a communication connection with the application client according to the request.
  • the way of establishing the connection may be a way similar to the three-way handshake process of the query and response of the network connection, or by saving the Internet Protocol address of the server.
  • Step 603 the cloud application client queries the type of vibration feedback supported by the associated physical device and the protocol version of the vibration feedback, and adds the queried information into the communication response message.
  • the type of vibration feedback mainly refers to the function or module of the physical device that can be used for vibration feedback, and the protocol version of vibration feedback specifies the specific modes that vibration feedback can perform.
  • vibration can include long vibration, short Vibration, long and short fusion vibration, and vibration of different frequencies and intensities, etc., and other types of vibration feedback are similar.
  • Step 604 the cloud application client sends the communication response message to the application client through the cloud application server.
  • Step 605 the application client generates vibration feedback information according to the received vibration feedback type and protocol version of the vibration feedback, and the recognized application scenario.
  • a communication connection for transmitting vibration feedback information is established between the application client on the server and the cloud application client, so that the vibration feedback can be sent from the server to the terminal device in time, and the server controls the terminal.
  • Vibration feedback provides the basis and improves the implementability of the scheme.
  • step S302 performing scene recognition on the cloud application corresponding to the cloud application client to obtain the scene recognition result, may include the following steps:
  • the cloud application scenario is identified according to the role attribute value, and the scenario identification result is obtained.
  • the scene recognition result may be determined according to an event that occurs in the target virtual character itself in the cloud application, or may be determined according to a time period in the cloud application.
  • the target virtual character is usually a character controlled by a user of the cloud application, or a character not controlled by the user that needs to be displayed.
  • the character attribute value may include data related to the application content such as the character's life value and stamina value, or data related to the operation of the application such as the character's spatial position in the application and the stage it has reached.
  • the scene of the cloud application can be identified, and the scene recognition result can be obtained. Specifically, if the attribute value of the character is higher or lower than a certain value, it can be identified as a specific scene, or if the attribute value of the character changes to a certain extent, it can also be identified as a scene that needs to trigger vibration.
  • the terminal device can be controlled to vibrate or flash at a certain frequency, thereby prompting the user and creating tense atmosphere.
  • performing scene recognition on the cloud application corresponding to the cloud application client to obtain the scene recognition result may include the following steps:
  • the scene identification of the target virtual character at the current moment is obtained, and the scene identification is used to represent the surrounding environment where the target virtual character is located;
  • the scene recognition result is determined according to a preset mapping relationship between scene identifiers and scene types, and the scene identifier at the current moment.
  • the mapping relationship between the time period of the cloud application and the scene identifier is preset.
  • the time period may refer to the progress of the cloud application, and the scene identifier is used to represent the surrounding environment where the target virtual character is located.
  • the target virtual character can be determined according to the time period. The scene ID at the current moment.
  • the time period may be a time period recorded by the cloud application from a specific time point, for example, a time period recorded when the virtual character enters a certain room in a game.
  • the application client can determine the scene recognition result.
  • the scene type is used to identify specific classifications of different scenes, and each scene type may correspond to multiple scene identifiers and require various vibration feedbacks.
  • the application client determines that the user's virtual character will fall into a trap in a certain game scene for too long, and the scene ID will identify the scene that falls into the trap.
  • the scene type corresponding to the scene identification is the falling type, therefore, it can be further determined according to the falling type that the corresponding scene recognition result is being injured.
  • the target scene type is used to identify the scene that needs vibration feedback. Therefore, if the scene recognition result does not belong to the target scene type, the physical device will not be controlled to perform vibration feedback. For example, if the scene type is a relatively mild scene such as recovery or rest, no vibration feedback will be performed, and the subsequent vibration feedback of the physical device will not be controlled.
  • a specific method for scene recognition is provided, so that the application scene can be targeted for recognition, which is conducive to improving the matching degree between the vibration feedback that needs to be performed later and the content of the current scene.
  • the above step of obtaining the character attribute value of the target virtual character in the cloud application at the current moment may include the following steps:
  • the above step, identifying the cloud application scenario according to the role attribute value, and obtaining the scenario recognition result may include the following steps:
  • the scene recognition result is determined according to the variation range of the attribute value.
  • the scene recognition result can be determined according to the user's operation on the target virtual character.
  • the game client receives the character control instruction for the target virtual character sent by the cloud game client.
  • the character control instruction may be action control such as movement, attack, or avoidance, or an action such as interaction control with other physics in the scene.
  • the game client controls the target virtual character in the cloud game according to the character control instruction, and also obtains the result of the control to obtain the character attribute value of the target virtual character at the current moment.
  • the movement of the target virtual character may cause the character to be attacked, resulting in a drop in health, so as to obtain the health of the target virtual character at the current moment.
  • the game client determines the change range of the attribute value according to the character attribute value of the target virtual character at the previous moment and the character attribute value at the current moment.
  • the previous moment refers to the moment before the game client controls the target virtual character. According to the comparison of character attribute values at two moments, the change range of attribute values can be determined.
  • the scene recognition result is determined according to the variation range of the attribute value. For example, if the life value is greatly reduced, it can be determined that the scene recognition result is a large amount of damage, so as to subsequently trigger the vibration of the physical device connected to the terminal to prompt the user.
  • the scene recognition result is determined through the change of the attribute value of the target virtual character, so that the scene recognition result can be determined in time during the cloud game process, the granularity of the scene recognition is improved, and the need for more accurate determination is made. Vibration feedback condition.
  • the vibration feedback information includes a vibration feedback type
  • the cloud application client controls the physical device to perform a vibration feedback operation indicated by the vibration feedback type.
  • vibration feedback types supported by different physical devices may be different.
  • mobile phones and other game peripherals may support different strengths and types of vibration effects.
  • the game client When generating the vibration feedback information, the game client generates the vibration feedback information according to the vibration feedback type supported by the recorded physical device and the vibration feedback required for the recognized scene.
  • the cloud game client controls the physical device to perform the vibration feedback operation indicated by the vibration feedback type when vibration feedback is required. For example, if the game client judges that the current scene requires one long vibration and multiple short vibrations, it generates vibration feedback information including the long vibration special effect type and the short vibration special effect type, and the cloud game client receives the vibration feedback information. , then control the phone to perform long vibration and short vibration.
  • the vibration feedback information includes a time parameter of vibration feedback, and the time parameter is used to indicate the time period for performing vibration feedback.
  • the cloud application client controls the physical device. During the time period indicated by the time parameter, the vibration feedback operation is performed.
  • the time parameter of the vibration feedback may include the start time and duration of the vibration, which may be determined according to the requirements of different scenarios.
  • the start time of the vibration is usually a relative time, which is set relative to the start time of the corresponding video stream, so as to determine when to start performing the vibration feedback operation.
  • the cloud game client starts to control the physical device to vibrate after playing the corresponding video stream for 10ms.
  • the duration of the vibration determines the end moment of the vibration feedback.
  • the vibration duration may not be included, but a pre-determined uniform duration may be used instead.
  • the duration of the vibration does not mean that the vibration feedback action needs to be continuously performed within the duration.
  • the vibration feedback action can be intermittent vibration, then within the duration, the physical device will vibrate intermittently, and the total duration reaches the duration. Limited value.
  • the vibration feedback information includes the time parameter of the vibration feedback, so that when the video stream is played, the start time and end time of the vibration feedback operation can be adjusted, so as to trigger the vibration feedback operation more flexibly and improve the flexibility of the solution .
  • the vibration feedback information includes the execution frequency of the vibration feedback
  • the cloud application client controls the physical device, and executes the vibration feedback operation according to the execution frequency of the vibration feedback .
  • the execution frequency of the vibration feedback specifies the frequency that the action involved in the vibration feedback needs to reach when the vibration feedback operation ends.
  • the execution frequency of vibration feedback can be determined according to the requirements of different scenarios, for example, different vibration execution frequencies can be set for different urgency.
  • the vibration execution frequency indicates the vibration frequency that needs to be reached when the vibration feedback ends, and the value range can be set as a relative value from 0 to 100.
  • the vibration feedback performed at the last moment may be continuous with the vibration feedback to be performed at the current moment. If the frequency of execution is different, the frequency can be adjusted so that it changes continuously without stopping and then restarting. . For example, if the vibration frequency at the last moment was 50, and the vibration frequency to be executed at the current moment is 70, then the vibration frequency can be evenly adjusted from 50 to 70 according to the preset step size.
  • the default execution frequency at the previous time is 0.
  • the vibration feedback information includes the execution frequency of vibration feedback, so that vibration feedback of different frequencies can be performed according to different scenes, thereby creating different atmospheres and improving the flexibility and expressiveness of special effects.
  • the vibration feedback information includes the execution intensity of the vibration feedback
  • the cloud application client controls the physical device, and executes the vibration feedback operation according to the execution intensity of the vibration feedback .
  • the execution intensity of the vibration feedback specifies the vibration intensity required for the action involved in the vibration feedback when the vibration feedback operation ends.
  • the execution intensity of the vibration feedback can be determined according to the requirements of different scenarios, for example, different vibration execution intensities can be set according to the severity.
  • the execution strength can be expressed as information such as amplitude, and the value range can be set as a relative value from 0 to 100.
  • the vibration feedback executed at the last moment may be continuous with the vibration feedback to be executed at the current moment. If the execution intensity is different, the intensity can be adjusted so that it changes continuously without stopping and restarting. conduct.
  • the vibration intensity at the previous moment was 50
  • the vibration intensity to be executed at the current moment is 70
  • the vibration intensity can be evenly adjusted from 50 to 70 according to the preset step size.
  • the default execution intensity at the previous time is 0.
  • the vibration feedback information includes the execution intensity of the vibration feedback, so that different intensities of vibration feedback can be performed according to different scene presentations, so as to be more consistent with the content of the game and increase the diversity of vibration feedback.
  • the above step S304 sending the video stream corresponding to the cloud application scene and the vibration feedback information to the cloud application client, includes:
  • the application client can run on a separate server, and the cloud application server is another server.
  • the server of the application client communicates with the cloud application server through a wired or wireless network.
  • the application client sends the rendered video stream corresponding to the cloud application scene and the generated vibration feedback information to the cloud application server, and the cloud application server packages the vibration feedback information according to the vibration protocol, and sends the packaging result and video stream to the cloud application server. to the cloud application client.
  • the cloud application server packages the vibration special effect into a data packet consisting of the start time, duration, vibration frequency and vibration intensity according to the above protocol, and then sends it to the cloud application client together with the video stream. In order to control the vibration effect.
  • the application client sends the video stream and vibration feedback information to the cloud application client through the cloud application server, so that multiple application clients can share the same cloud application server, thereby reducing the operating cost of the cloud application .
  • FIG. 7 is a flowchart of a cloud application-based device control method provided by another embodiment of the present application, including the following step:
  • Step S701 establishing a communication connection with the cloud application server
  • Step S702 receiving the video stream and vibration feedback information corresponding to the cloud application scene sent by the cloud application server, the vibration feedback information is generated by the application client on the server according to the cloud application scene;
  • Step S703 playing the video stream corresponding to the cloud application scene, and controlling the physical device associated with the cloud application client to perform a vibration feedback operation according to the vibration feedback information.
  • the cloud application client establishes a communication connection with the cloud application server.
  • the communication establishment process is usually initiated by the application client.
  • the cloud application client starts the cloud application after logging into the cloud application platform, and the cloud application platform starts the corresponding application client on the server, and the application client actively establishes communication with the cloud application client.
  • the cloud application client receives the communication establishment request sent by the application client, and saves it locally for subsequent sending of control information and various response messages.
  • the cloud application client After the communication is established, the cloud application client will receive the video stream and vibration feedback information corresponding to the cloud application scene from the cloud application server.
  • the progress of the game in cloud games is controlled by the cloud game client, and the game client on the server executes the game content according to the user's operation, and renders the game scene of the cloud game into a video stream, and also Generate vibration feedback information according to the content of the game scene.
  • the cloud game client will play the video stream of the cloud game scene to the user, so as to show the game screen and operation results to the user for subsequent games, and will control the physical device associated with the cloud game client to perform vibration feedback operations.
  • the mobile phone is controlled to vibrate according to the vibration information in the vibration feedback information.
  • the cloud application client establishes a communication connection with the cloud application server, and displays the application content and performs vibration feedback operations according to the received video stream and vibration feedback information, so that it can be implemented in the cloud application scenario.
  • the application client on the server directly triggers the vibration of the physical device associated with the cloud application client, avoiding the situation that the vibration does not match the application content when the vibration is performed according to the audio or video content, and improves the accuracy of the vibration feedback.
  • the above steps, establishing a communication connection with the cloud application server may include the following steps:
  • the cloud application server sending a communication response message to the cloud application server, where the communication response message includes the vibration feedback type and the protocol version of the vibration feedback, so that the application client protocol version to generate the vibration feedback information.
  • the cloud game client receives a communication establishment request sent by the cloud game server.
  • the communication establishment request usually includes the communication parameters required for communication, and the cloud game client establishes a communication connection with the cloud game server according to the communication establishment request.
  • the communication establishment request will also include a special effect query instruction, and the cloud game client will query the type of vibration feedback and the protocol version of the vibration feedback supported by the physical device associated with the cloud game client according to the query instruction.
  • the vibration feedback type is mainly used to identify the type of vibration that the physical device can perform, such as vibration length, mode, and so on.
  • the protocol version is mainly used to specify the parameters and communication formats that need to be used when defining the vibration feedback operation.
  • the cloud game client sends a communication response message to the cloud game server.
  • the communication response message includes the vibration feedback type and the protocol version of the vibration feedback.
  • the communication response message is used to enable the cloud game server to establish a communication connection with the cloud game client.
  • the communication response message may carry the communication parameters of the cloud game client, such as an Internet address, for the cloud game server to establish a corresponding communication connection.
  • the physical device associated with the cloud application client further includes a third-party device; according to the vibration feedback information, the physical device associated with the cloud application client is controlled to execute Vibration feedback operation, including:
  • vibration feedback information generate vibration control instructions for third-party equipment
  • a vibration control command is sent to the third-party device, so that the third-party device performs a vibration feedback operation according to the vibration feedback information.
  • Third-party devices refer to devices other than those used for application control and cloud application content display.
  • the third-party device may be a material network device and communicates with the terminal device where the cloud application client is located through the network.
  • the cloud application client generates vibration control instructions for third-party devices according to the vibration feedback information.
  • the vibration control instruction includes parameters for performing vibration generated according to the vibration feedback information. For example, values such as the start and end times of the vibration, presentation mode, and intensity.
  • the cloud application client sends a vibration control instruction to the third-party device, so that the third-party device performs a vibration feedback operation according to the vibration feedback information.
  • the vibration feedback information sent by the game client to the cloud game client may include information about seat shaking control , and the cloud game client generates a vibration control instruction for seat shaking control according to the vibration feedback information, and sends it to the smart seat in the house, and the smart seat shakes according to the received vibration control instruction. So as to give users a more immersive feeling.
  • the cloud game client also controls a third-party device to coordinate vibration feedback, thereby enriching the vibration display process and improving the vibration display effect.
  • Fig. 8A schematically shows a composition block diagram of a cloud application-based device control apparatus in an embodiment of the present application, which is used to execute processing related to the application client on the server in the above embodiments.
  • the device control device 800 includes:
  • a communication establishment module 801 configured to establish a communication connection with a cloud application client on the terminal device, and the cloud application client is used to present the received cloud application video stream;
  • the scene identification module 802 is configured to perform scene identification on the cloud application corresponding to the cloud application client to obtain a scene identification result, the scene identification result including media content associated with the identified cloud application scene;
  • a vibration feedback information generation module 803, configured to generate vibration feedback information corresponding to the cloud application scene according to the scene recognition result;
  • a vibration feedback information sending module 804 configured to send the video stream corresponding to the cloud application scene and the vibration feedback information to the cloud application client, so that the cloud application client presents the video of the cloud application scene During streaming, according to the vibration feedback information, control the physical device associated with the cloud application client to perform a vibration feedback operation.
  • the communication establishment module 801 includes:
  • a communication establishment request sending unit configured to send a communication establishment request to the cloud application client
  • a communication response message receiving unit configured to receive a communication response message returned by the cloud application client; according to the communication response message, establish a communication connection with the cloud application client;
  • the communication response message includes the type of vibration feedback supported by the physical device and the protocol version of vibration feedback;
  • the scene recognition module 802 is configured to generate the vibration feedback information according to the scene recognition result, the vibration feedback type, and the protocol version of the vibration feedback.
  • the scene recognition module 802 includes:
  • a role attribute value acquisition unit configured to acquire the role attribute value of the target virtual role in the cloud application at the current moment
  • a scenario identification unit configured to identify the cloud application scenario according to the role attribute value, and obtain the scenario identification result.
  • the scene recognition module 802 includes:
  • the scene identification acquisition unit is used to obtain the scene identification of the target virtual character at the current moment according to the mapping relationship between the preset time period and the scene identification, and the scene identification is used to indicate the surrounding area where the target virtual character is located environment;
  • the scene recognition result determination unit is configured to determine the scene recognition result according to the preset mapping relationship between scene identifiers and scene types, and the scene identifier at the current moment.
  • the role attribute value acquisition unit includes:
  • a role control instruction receiving subunit configured to receive a role control instruction for the target virtual character sent by the cloud application client
  • the virtual character control subunit is configured to control the target virtual character according to the character control instruction, and obtain the character attribute value of the target virtual character at the current moment;
  • the scene recognition unit includes:
  • the attribute value change range determination subunit is used to determine the attribute value change range according to the character attribute value of the target virtual character at the previous moment and the character attribute value at the current moment;
  • the recognition result determination subunit is configured to determine the scene recognition result according to the variation range of the attribute value.
  • the vibration feedback information includes a vibration feedback type
  • the cloud application client controls the physical device to perform a vibration feedback operation indicated by the vibration feedback type.
  • the vibration feedback information includes a vibration feedback time parameter
  • the time parameter is used to indicate the time period for vibration feedback
  • the cloud application client controls the physical The device performs a vibration feedback operation within the time period indicated by the time parameter.
  • the vibration feedback information includes the execution frequency of vibration feedback
  • the cloud application client controls the physical device to execute the vibration feedback execution frequency according to the vibration feedback execution frequency. Vibration feedback operation described above.
  • the vibration feedback information includes the execution intensity of the vibration feedback
  • the cloud application client controls the physical device, and executes the vibration feedback execution intensity according to the vibration feedback execution intensity. Vibration feedback operation described above.
  • the vibration feedback information sending module 804 includes:
  • the server communication unit is configured to send the video stream corresponding to the cloud application scene and the vibration feedback information to the cloud application server, so that the cloud application server packages the vibration feedback information according to the vibration protocol, and sends the packaged result and sending the video stream to the cloud application client.
  • a device control device based on a cloud application is provided.
  • Fig. 8B schematically shows a composition block diagram of a cloud application-based device control apparatus in another embodiment of the present application, which is used to execute processes related to the cloud application client in the above-mentioned embodiments.
  • the device control device 810 includes:
  • a communication connection establishment module 811 configured to establish a communication connection with the cloud application server
  • the vibration feedback information receiving module 812 is configured to receive the video stream and vibration feedback information corresponding to the cloud application scene sent by the cloud application server, and the vibration feedback information is generated by the application client on the server according to the cloud application scene ;
  • Play module 813 for playing the video stream corresponding to the cloud application scene
  • the vibration feedback execution module 814 is configured to control the physical device associated with the cloud application client to perform a vibration feedback operation according to the vibration feedback information.
  • the communication connection establishment module 811 includes:
  • a communication establishment request receiving unit configured to receive the communication establishment request sent by the cloud application server
  • a vibration feedback query unit configured to query the type of vibration feedback supported by the physical device and the protocol version of the vibration feedback according to the communication establishment request;
  • a communication response message sending unit configured to send a communication response message to the cloud application server, where the communication response message includes the vibration feedback type and the protocol version of the vibration feedback, so that the application client can
  • the vibration feedback type and the protocol version of the vibration feedback are used to generate the vibration feedback information.
  • the physical device associated with the cloud application client further includes a third-party device;
  • the vibration feedback execution module further includes:
  • a vibration control instruction generation module which generates a vibration control instruction for the third-party device according to the vibration feedback information
  • a vibration control instruction sending module configured to send the vibration control instruction to the third-party device, so that the third-party device performs a vibration feedback operation according to the vibration feedback information.
  • FIG. 9 shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present application.
  • the electronic device 900 includes a central processing unit (Central Processing Unit, CPU) 901, which can be stored in a program in a read-only memory (Read-Only Memory, ROM) 902 or loaded from a storage part 908 to a random Various appropriate actions and processes are executed by accessing programs in the random access memory (RAM) 903 .
  • RAM 903 random access memory
  • various programs and data necessary for system operation are also stored.
  • the CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904.
  • An input/output (Input/Output, I/O) interface 905 is also connected to the bus 904 .
  • the following components are connected to the I/O interface 905: an input part 906 including a keyboard, a mouse, etc.; an output part 907 including a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (Liquid Crystal Display, LCD), etc., and a speaker ; the storage part 908 including hard disk etc.; and the communication part 909 including the network interface cards such as LAN (Local Area Network, local area network) card, modem etc.
  • the communication section 909 performs communication processing via a network such as the Internet.
  • a drive 910 is also connected to the I/O interface 905 as needed.
  • a removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 910 as necessary so that a computer program read therefrom is installed into the storage section 908 as necessary.
  • the processes described in the various method flowcharts can be implemented as computer software programs.
  • the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts.
  • the computer program may be downloaded and installed from a network via communication portion 909 and/or installed from removable media 911 .
  • this computer program is executed by a central processing unit (CPU) 901, various functions defined in the system of the present application are performed.
  • CPU central processing unit
  • the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
  • Computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable one of the above The combination.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program codes are carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. .
  • Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that includes one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block in the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a A combination of dedicated hardware and computer instructions.
  • the example implementations described here can be implemented by software, or can be implemented by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, server, touch terminal, or network device, etc.) execute the method according to the embodiment of the present application.
  • a non-volatile storage medium which can be CD-ROM, U disk, mobile hard disk, etc.
  • a computing device which may be a personal computer, server, touch terminal, or network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • User Interface Of Digital Computer (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本申请提供一种基于云应用的设备控制方法、装置、电子设备和可读介质。该方法包括：建立与终端设备上的云应用客户端之间的通信连接，所述云应用客户端用于呈现所接收到的云应用视频流；对所述云应用客户端所对应的云应用进行场景识别，得到场景识别结果，所述场景识别结果包括与识别到的云应用场景相关联的媒体内容；根据所述场景识别结果，生成与所述云应用场景所对应的振动反馈信息；及，向所述云应用客户端发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用客户端在呈现所述云应用场景的视频流时，根据所述振动反馈信息，控制与所述云应用客户端相关联的物理设备执行振动反馈操作。

Description

基于云应用的设备控制方法、装置、电子设备和可读介质
本申请要求于2021年7月21日提交中国专利局、申请号为202110827404.8、申请名称为“基于云游戏的设备控制方法、装置、电子设备和可读介质”的中国专利申请的优先权。
技术领域
本申请涉及计算机技术领域，尤其涉及一种基于云应用的设备控制方法、装置、电子设备和可读介质。
发明背景
随着互联网技术的迅速发展，人们对娱乐形式的要求越来越高。游戏就是其中重要的娱乐形式，特别是对于当下热门的云游戏，不需要用户拥有高配置的硬件就可以为用户提供优秀的娱乐体验。
对于这种类型的云应用，由于云应用的运行过程发生在服务器上，服务器上的应用客户端需要将应用的画面和场景渲染成视频发送给用户终端，在用户终端上播放来向展示界面内容。
但是，由于云应用运行在服务器上，终端设备上仅能接收到渲染后的音视频流，所能提供的互动方式有限，难以提供丰富、沉浸的体验，影响了云应用与用户之间的互动效果。
发明内容
基于上述技术问题，本申请提供一种基于云应用的设备控制方法、装置、电子设备和可读介质，以能够在云应用的场景下，由服务器触发与应用内容相符合的振动反馈，从而提升振动反馈的准确性，从而提升用户体验。
本申请的其他特性和优点将通过下面的详细描述变得显然，或部分地通过本申请的实践而习得。
根据本申请实施例的一个方面，提供一种基于云应用的设备控制方法，由电子设备执行，包括：
建立与终端设备上的云应用客户端之间的通信连接，所述云应用客户端用于呈现所接收到的云应用视频流；
对所述云应用客户端所对应的云应用进行场景识别，得到场景识别结果，所述场景识别结果包括与识别到的云应用场景相关联的媒体内容；
根据所述场景识别结果，生成与所述云应用场景所对应的振动反馈信息；及，
向所述云应用客户端发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用客户端在呈现所述云应用场景的视频流时，根据所述振动反馈信息，控制与所述云应用客户端相关联的物理设备执行振动反馈操作。
根据本申请实施例的另一个方面，提供一种基于云应用的设备控制装置，包括：
通信建立模块，用于建立与终端设备上的云应用客户端之间的通信连接，所述云应用客户端用于呈现所接收到的云应用视频流；
场景识别模块，用于对所述云应用客户端所对应的云应用进行场景识别，得到场景识别结果，所述场景识别结果包括与识别到的云应用场景相关联的媒体内容；
振动反馈信息生成模块，用于根据所述场景识别结果，生成与所述云应用场景所对应 的振动反馈信息；及，
振动反馈信息发送模块，用于向所述云应用客户端发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用客户端在呈现所述云应用场景的视频流时，根据所述振动反馈信息，控制与所述云应用客户端相关联的物理设备执行振动反馈操作。
根据本申请实施例的另一个方面，提供一种基于云应用的设备控制方法，由电子设备执行，所述方法包括：
建立与云应用服务器之间的通信连接；
接收所述云应用服务器发送的云应用场景对应的视频流和振动反馈信息，所述振动反馈信息是由服务器上的应用客户端根据所述云应用场景生成的；及，
播放所述云应用场景对应的视频流，并根据所述振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作。
根据本申请实施例的另一个方面，提供一种基于云应用的设备控制装置，包括：
通信连接建立模块，用于建立与云应用服务器之间的通信连接；
振动反馈信息接收模块，用于接收所述云应用服务器发送的云应用场景对应的视频流和振动反馈信息，所述振动反馈信息是由服务器上的应用客户端根据所述云应用场景生成的；
播放模块，用于播放所述云应用场景对应的视频流；及，
振动反馈执行模块，用于根据所述振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作。
根据本申请实施例的一个方面，提供一种电子设备，该电子设备包括：处理器；以及存储器，用于存储处理器的可执行指令；其中，该处理器配置为经由执行可执行指令来执行如以上技术方案中的基于云应用的设备控制方法。
根据本申请实施例的一个方面，提供一种计算机可读存储介质，其上存储有计算机程序，当该计算机程序被处理器执行时，实现如以上技术方案中的基于云应用的设备控制方法。
根据本申请实施例的一个方面，提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述各种可选实现方式中提供基于云应用的设备控制方法。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。
附图简要说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：
图1是本申请实施例中的一种基于云应用的设备控制系统的示例网络架构图；
图2为本申请实施例中的一种云应用设备控制框架的示意图；
图3为本申请中振动特效设计流程的示意图；
图4为本申请实施例提供的一种基于云应用的设备控制方法的流程图；
图5为本申请实施例中触发振动反馈的过程；
图6为本申请实施例中与云应用客户端建立通信连接的过程；
图7为本申请另一实施例提供的基于云应用的设备控制方法的流程图；
图8A示意性地示出了本申请实施例中基于云应用的设备控制装置的组成框图；
图8B示意性地示出了本申请另一实施例中基于云应用的设备控制装置的组成框图；
图9示出了适于用来实现本申请实施例的电子设备的结构示意图。
实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本申请将更加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。
此外，所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施例中。在下面的描述中，提供许多具体细节从而给出对本申请的实施例的充分理解。然而，本领域技术人员将意识到，可以实践本申请的技术方案而没有特定细节中的一个或更多，或者可以采用其它的方法、组元、装置、步骤等。在其它情况下，不详细示出或描述公知方法、装置、实现或者操作以避免模糊本申请的各方面。
附图中所示的方框图仅仅是功能实体，不一定必须与物理上独立的实体相对应。即，可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
附图中所示的流程图仅是示例性说明，不是必须包括所有的内容和操作/步骤，也不是必须按所描述的顺序执行。例如，有的操作/步骤还可以分解，而有的操作/步骤可以合并或部分合并，因此实际执行的顺序有可能根据实际情况改变。
本申请各实施例应用于云应用的应用场景。所谓的“云应用”，是指云计算技术在应用层的体现。“云应用”的工作原理是把传统应用程序(或软件)“本地安装、本地运算”的使用方式，变为“即取即用”的服务，通过互联网或局域网连接并操控远程服务器集群，完成业务逻辑或运算任务。
“云应用”的主要载体为互联网技术，云应用客户端是以瘦客户端(Thin Client)或智能客户端(Smart Client)为展现形式，其界面是HTML5、Javascript或Flash等技术的集成。云应用不但可以帮助用户降低IT成本，更能大大提高工作效率。
具体地，基于云应用的设备控制系统可以运行于如图1所示的网络架构中。如图1所示，基于云应用的设备控制系统包括云端服务器101以及多种类型的终端设备102-106，云端服务器101可以提供各类终端设备102-106的云应用的控制过程，即根据用户在终端设备102-106侧的控制操作向云端服务器101发送的操作指令，云端服务器101针对操作指令进行云应用实例的控制、场景识别、操作结果计算以及音视频渲染等处理，向终端设备102-106返回操作指令对应的信息，例如画面、音乐等。
可以理解的是，图1中示出了多种终端设备102-106，终端设备可以为计算机设备，在实际场景中可以有更多或更少种类的终端设备参与到云应用的控制的过程中，具体数量和种类因实际场景而定，此处不做限定。
另外，图1中示出了一个云端服务器101，但在实际场景中，也可以有多个云端服务器的参与，特别是在多个不同的云游戏实例运行的场景中，具体云端服务器数量因实际场景而定。
云端服务器可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN、以及大数据和人工智能平台等基础云计算服务的云端服务器。终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表等，但并不局限于此。终端以及云端服务器可以通过有线或无线通信方式进行直接或间接地连接，终端以及云端服务器可以连接组成区块链网络，本申请在此不做限制。
在各类具体的云应用中，云游戏(Cloud gaming)是一种以云计算技术为基础的在线游戏技术，也可称为游戏点播(gaming on demand)。云游戏技术使图形处理与数据运算能力相对有限的轻端设备(如瘦客户端thin client)能运行高品质游戏。
应理解，本申请的技术方案可以应用于云游戏场景，例如云游戏平台等，包括云端服务器和游戏终端设备，其中，云端服务器中包含云游戏的控制功能的设备系统或程序，游戏终端用于在用户游戏的过程中控制用户的各种操作。
在云游戏场景下，游戏并不在玩家的游戏终端设备运行，而是在云端服务器中运行，并由云端服务器将游戏场景渲染为音视频流，通过网络传输给玩家的游戏终端设备。玩家的游戏终端设备无需拥有强大的图形运算与数据处理能力，仅需拥有基本的流媒体播放能力，以及获取玩家输入指令并发送给云端服务器的能力即可。
根据本申请实施例中，当云应用为云游戏时，基于云游戏的设备控制方法可以运行于个人移动终端，也可以运行于云端服务器，还可以运行于第三方设备以提供云游戏的控制。具体的云游戏的设备控制系统可以是以程序的形式在上述设备中运行，也可以作为上述设备中的系统部件进行运行，还可以作为云端服务程序的一种，具体运作模式因实际场景而定，此处不做限定。
本申请提出一种基于云应用的设备控制方法，该方法可以应用于图2所示的云应用设备控制框架中。图2为本申请实施例中的一种云应用设备控制框架的示意图，包括服务器上的应用客户端210、云应用服务器220和终端设备上的云应用客户端230。
其中，应用客户端210通过终端识别模块211，向云应用服务器220下发指令，指令最终到达云应用客户端230，以便查询云应用客户端230当前所在的终端设备(如智能终端)或者与云应用客户端230相关联的物理设备，是否支持振动反馈，以及支持的振动反馈的格式内容。
终端控制模块213根据终端识别模块211提供的信息，与云应用客户端230建立通信，该通信通过云应用服务器220中转，以通过建立的通信下发振动反馈信息。
应用客户端210的振动反馈控制模块212，根据应用场景的不同，触发相应的振动反馈，获取振动反馈信息的实际文件，并通过终端控制模块213，将该文件下发到云应用服务器220，由云应用服务器220中转，最终下发到云应用客户端230。
云应用服务器220通过终端通信模块221与云应用客户端230建立链接，从云应用客户端230获取振动反馈的支持信息，以及向云应用客户端230下发振动反馈的控制协议。振动协议打包模块222用于接收应用客户端210下发的振动反馈信息，并根据定义好的协议，将该振动反馈信息打包，并通过终端通信模块221传送到云应用客户端230。
云应用客户端230通过通信模块231，根据云应用服务器220的请求，与其建立通信连接，并且接收云应用服务器220下发的硬件查询指令和振动反馈指令等信息。振动反馈解析模块232用于在接收到通信模块231的振动反馈指令后，根据定义好的协议进 行解析，并调用硬件控制模块233。硬件控制模块233用于接收振动反馈解析模块232的指令，调用终端设备的对应接口进行实际的硬件驱动，以实现振动反馈。
具体地，当云应用为云游戏时，应用客户端210指游戏客户端，云应用服务器220指云游戏服务器，云应用客户端230指云游戏客户端。
为了结合应用场景触发振动反馈，特效设计通常需要根据应用场景展示专属的效果。具体地，为了便于介绍，请参阅图3，图3为本申请实施例中振动特效设计流程的示意图，包括了服务器上的应用客户端310、云应用服务器320和终端设备上的云应用客户端330之间的交互。
其中，效果设计311是指，服务器或者服务器上的应用客户端310为特定的场景，设计专属的振动反馈效果。
场景识别312是指，在应用(如游戏)运行过程中，服务器或者应用客户端310识别出相应的场景，并匹配出相关的特效描述文件。比如在射击类游戏中，用户使用不同的枪械可以触发不同的振动反馈；用户的虚拟角色被击中，根据掉血量不同，触发不同强度的振动，让用户通过振动就可以直接感知到掉血量；用户在遭遇到敌人时，触发高频短促的振动，提示用户危险来临。
振动触发313是指，服务器上的应用客户端310下发振动请求，以请求调用物理设备的振动触发接口。但是，因为云应用是在云应用服务器320渲染，此请求无法直接送达物理设备，因此需要经由云应用服务器320进行中转。
透传终端321是指，云应用服务器320收到应用客户端310下发的振动请求后，通过通讯连接透传到云应用客户端330。
系统触发331是指，云应用客户端330收到云应用服务器320透传的振动请求后，调用物理设备的接口来触发终端设备上硬件的振动。
可以理解的是，本申请所提供的方法可作为一种程序写入，作为硬件系统中的一种处理逻辑，也可以作为一种云应用的控制装置，采用集成或外接的方式实现上述处理逻辑。
结合上述流程架构，下面将对本申请中基于云应用的设备控制方法进行介绍，请参阅图4，图4为本申请实施例提供的一种基于云应用的设备控制方法的流程图，由电子设备执行，例如，由图2中服务器上的应用客户端210来执行，包括以下步骤：
步骤S301，建立与终端设备上的云应用客户端之间的通信连接，云应用客户端用于呈现所接收到的云应用视频流。
在本实施例中，服务器上的应用客户端与终端设备上的云应用客户端，建立通信连接。应用客户端为运行在服务器上的具体应用程序，而云应用客户端为终端设备上用于呈现所接收到的云应用视频流的应用。
具体地，当应用客户端为游戏客户端时，云应用客户端为云游戏客户端。例如，以赛车游戏为例，云游戏客户端具体可以是手机上浏览器、播放器或者专用的客户端，主要用于播放服务器提供的游戏画面，而游戏客户端则是在服务器上运行的赛车游戏的实际应用程序。
本实施例中的通信连接主要用于传输振动反馈相关的信息，该通信连接可以是专用的通信信道，也可以同时用于应用客户端与云应用客户端之间的视频流通信以及控制信息通信。
步骤S302，对云应用客户端所对应的云应用进行场景识别，得到场景识别结果，场 景识别结果包括与识别到的云应用场景相关联的媒体内容。
在本实施例中，云应用实际上由服务器上的应用客户端运行。应用客户端会根据云应用的执行情况以及用户的操作，对云应用所处的场景进行识别。例如，游戏场景通常包括游戏进度、游戏画面、游戏数据等内容，游戏在进行过程中的各种数据的变化均有可能导致游戏场景的变化，例如，游戏中虚拟角色所处的环境变化、生命值等属性发生变化、虚拟角色遭遇各种事件或者与环境互动等。
应用客户端可以根据云应用的数据内容和数据变化来识别到具体的云应用场景，从而得到场景识别结果。场景识别结果包括与识别到的云应用场景相关联的媒体内容，用于指示云应用在接下来的一段时间段内虚拟角色的表现。例如，在云游戏中，虚拟角色从高处掉落，则场景识别结果可以指示，在接下来的一个时间段中，虚拟角色掉落在地面上、以及由于高处掉落受到的伤害。
步骤S303，根据场景识别结果，生成与云应用场景所对应的振动反馈信息。
具体地，场景识别结果中所标识出的云应用场景，可以确定在接下来的一个时间段云应用中将会发生的变化或者事件，从而可以确定在该云应用场景中需要触发的振动反馈，进而生成与识别到的云应用场景所对应的振动反馈信息。振动反馈信息包括所需要进行的振动反馈的种类以及执行振动反馈的参数，例如执行时间等。
取决于与云应用客户端相关联的物理设备支持的振动反馈的种类，振动反馈信息中可以包括一种或多种振动反馈对应的参数。
取决于云应用场景的具体内容，振动反馈信息中可以包括进行多次振动反馈所需要的参数。例如，若云游戏场景中，目标角色在下落期间会与障碍物发生三次碰撞，而每次碰撞均需要进行振动反馈，则振动反馈信息中将会包括对应于三次碰撞的三组振动的振动反馈参数。
步骤S304，向云应用客户端发送云应用场景对应的视频流以及振动反馈信息，以使云应用客户端在呈现云应用场景的视频流时，根据振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作。
具体地，为了便于介绍，请参阅图5，图5为本申请实施例中触发振动反馈的过程，包括如下步骤：
步骤501，应用客户端通过云应用服务器向云应用客户端发送云应用场景对应的视频流以及振动反馈信息。
其中，云应用场景的视频流是根据云应用场景中将会展现的应用画面渲染出的视频流。
步骤502，云应用客户端在接收到视频流后，向用户播放该视频流。
本步骤中，云应用客户端向用户展示视频内容，例如用户操作所发生的结果。
步骤503，在播放视频流时，云应用客户端还会根据接收到的振动反馈信息，控制相关联的物理设备执行振动反馈操作。
其中，与云应用客户端相关联的物理设备可以是云应用客户端所处的智能终端。
例如，云应用为竞速类游戏，云应用客户端所处的智能终端为手机。在该竞速类游戏中，用户在手机上操控所驾驶的车辆向对手冲撞，则服务器上的游戏客户端所发送的云游戏场景的视频流中会展现车辆撞击对手的画面，而在视频流播放到车辆撞击到对手时，云游戏客户端还会控制手机进行1秒钟的振动，从而展现撞击的冲击力，提升用户的体验和沉浸感。
可见，通过振动反馈操作，配合所呈现的云应用场景的视频流，能够达到更加立体的 游戏体验。
步骤504，物理设备完成振动反馈后，向云应用客户端返回振动的执行结果。
步骤505，云应用客户端生成包括该振动的执行结果的振动反馈响应消息，并将振动反馈响应消息通过云应用服务器发送给应用客户端。
在本申请的实施例中，通过场景识别得到场景识别结果，生成云应用场景对应的振动反馈信息，并且通过与云应用客户端之间的通信连接，向云应用客户端发送振动反馈信息，以控制相关联的物理设备进行振动反馈操作。通过上述的方式，能够获得如下好处：
1)在云应用的场景下，通过打通云应用的全链路，实时下发振动的驱动信号，其延时和云应用的视频流保持一致，从而让用户体验到零延时的振动，提升了振动反馈的精准性，从而提升用户体验。
2)基于对场景的识别，设计相应的振动反馈信息，可以更加贴合用户的实际操作，对用户的操作和体验形成助力，使得用户更加沉浸于应用内容(例如游戏内容)本身，同时还能够最大化地发挥物理设备的硬件优势。
在本申请一个实施例中，上述步骤S301，建立与终端设备上的云应用客户端之间的通信连接，可以包括如下步骤：
向所述云应用客户端发送通信建立请求；
接收所述云应用客户端返回的通信响应消息；
根据所述通信响应消息，与所述云应用客户端建立通信连接；
其中，所述通信响应消息中包含所述物理设备所支持的振动反馈类型和振动反馈的协议版本；
所述根据所述场景识别结果，生成与所述云应用场景所对应的振动反馈信息，包括：
根据所述场景识别结果、以及所述振动反馈类型和所述振动反馈的协议版本，生成所述振动反馈信息。
具体地，服务器上的应用客户端向云应用客户端发送通信建立请求。该通信建立请求可以基于应用客户端与云应用客户端之间已经建立的音视频通信链路或者控制通信链路来发送，并且可以在正式启动云应用之前的任一时刻进行。例如，在云应用客户端登录账户时或者在云应用客户端建立其他通信连接时，一同建立用于振动反馈的通信连接。
具体地，为了便于介绍，请参阅图6，图6为本申请实施例中与云应用客户端建立通信连接的过程，包括如下步骤：
步骤601，应用客户端根据已经登录到服务器的云应用客户端的列表，通过云应用服务器，向云应用客户端发送通信建立请求。
步骤602，云应用客户端在接收到该通信建立请求时，根据该请求与应用客户端建立通信连接。
具体地，建立连接的方式可以是类似于网络连接的问询和应答的三握手过程等方式，或者通过保存服务器的互联网协议地址等方式进行。
步骤603，云应用客户端查询相关联的物理设备所支持的振动反馈类型和振动反馈的协议版本，并且将所查询到的信息加入到通信响应消息中。
其中，振动反馈类型主要指的是物理设备所具有的可以用于进行振动反馈的功能或者模块，而振动反馈的协议版本则规定了振动反馈可以执行的具体模式，例如振动可以包括长振动、短振动、长短融合振动，以及不同频率和强度的振动等，其他类型的振动反馈也是类似。
步骤604，云应用客户端将通信响应消息通过云应用服务器发送给应用客户端。
步骤605，应用客户端根据接收到的振动反馈类型和振动反馈的协议版本，以及识别到的应用场景，生成振动反馈信息。
在本申请的实施例中，服务器上的应用客户端与云应用客户端之间建立用于传送振动反馈信息的通信连接，使得振动反馈可以及时地从服务器发送到终端设备，为服务器控制终端进行振动反馈提供基础，提高了方案的可实施性。
在本申请一个实施例中，上述步骤S302，对云应用客户端所对应的云应用进行场景识别，得到场景识别结果，可以包括如下步骤：
获取所述云应用中的目标虚拟角色在当前时刻下的角色属性值；
根据所述角色属性值，识别所述云应用场景，得到所述场景识别结果。
具体地，取决于云应用类型的不同，场景识别结果可以根据云应用中目标虚拟角色本身所发生的事件确定，也可以根据云应用中的时间段来确定。
目标虚拟角色通常是云应用的用户所控制的角色，或者是需要进行展示的非用户控制的角色等。角色属性值可以包括角色的生命值、体力值等与应用内容相关的数据，或者是角色在应用中所处的空间位置、所进行到的阶段等与应用运行相关的数据。
根据目标虚拟角色在当前时刻下的角色属性值，可以识别云应用的场景，得到场景识别结果。具体地，角色属性值高于或者低于一定数值，可以被识别为某个特定场景，或者角色的属性值发生一定幅度的变化，也可以被识别为需要触发振动的场景。
例如，对于角色生命值，在生命值低于设定阈值的情况下，可以识别到生命值过低场景，在该场景中，可以控制终端设备按照一定频率进行振动或者闪烁，从而提示用户并且营造紧张氛围。
或者，上述步骤S302，对云应用客户端所对应的云应用进行场景识别，得到场景识别结果，可以包括如下步骤：
根据预设的时间段与场景标识之间的映射关系，获取目标虚拟角色在当前时刻下的场景标识，所述场景标识用于表示所述目标虚拟角色所处的周边环境；
根据预设的场景标识与场景类型之间的映射关系、以及所述在当前时刻下的场景标识，确定所述场景识别结果。
其中，预先设置云应用所处的时间段与场景标识之间的映射关系。时间段可以指的是云应用所处的进度，而场景标识用于表示所述目标虚拟角色所处的周边环境。
具体地，云应用的场景可以与其进行的进度(即时间长度)之间存在关联关系。例如，对于跑酷类这样的游戏，游戏进度随着时间进行推移，在达到不同的时间段时，就会到达不同的游戏场景，对应于不同的场景标识，根据时间段则可以确定目标虚拟角色在当前时刻下的场景标识。
或者，时间段可以是云应用的从一个特定时间点开始记录的时间段，例如，虚拟角色在游戏中进入某个房间后开始计时所记录下的时间段。
根据预设的场景标识与场景类型之间的映射关系，应用客户端可以确定场景识别结果。具体地，场景类型用于标识不同场景的具体分类，每个场景类型可以对应多个场景标识并且需要进行各种不同的振动反馈。
例如，应用客户端根据时间段与场景标识之间的映射关系，确定用户的虚拟角色在某个游戏场景内时间过长而将会落入陷阱，场景标识会标识出落入陷阱的场景。该场景标识所对应的场景类型为下落类型，因此，可以进一步根据下落类型确定对应的场景识别结果 为受到伤害。
此外，目标场景类型用于标识出需要进行振动反馈的场景，因此，若场景识别结果不属于目标场景类型，则后续不会控制物理设备进行振动反馈的动作。例如，场景类型为恢复或者休息等相对缓和的场景，则不进行振动反馈，也不会后续对物理设备的振动反馈进行控制。
在本申请的实施例中，提供了进行场景识别的具体方式，从而能够对应用场景进行针对性的识别，有利于提升后续需要进行的振动反馈与当前场景内容之间的匹配度。
在本申请一个实施例中，上述步骤，获取所述云应用中的目标虚拟角色在当前时刻下的角色属性值，可以包括如下步骤：
接收所述云应用客户端发送的针对所述目标虚拟角色的角色控制指令；
根据所述角色控制指令，对所述目标虚拟角色进行控制，得到所述目标虚拟角色在所述当前时刻下的角色属性值；
上述步骤，根据所述角色属性值，识别所述云应用场景，得到所述场景识别结果，可以包括如下步骤：
根据所述目标虚拟角色在前一时刻下的角色属性值以及在所述当前时刻下的角色属性值，确定属性值变化幅度；
根据所述属性值变化幅度，确定所述场景识别结果。
在本实施例中，场景识别结果可以根据用户对目标虚拟角色的操作来确定。具体地，以游戏客户端和云游戏客户端为例，游戏客户端接收云游戏客户端发送的针对目标虚拟角色的角色控制指令。角色控制指令可以是移动、攻击或者回避等动作控制，或者与场景内的其他物理的互动控制等动作。
游戏客户端根据角色控制指令，在云游戏中对目标虚拟角色进行控制，并且还会得到控制所产生的结果，以得到目标虚拟角色在当前时刻下的角色属性值。例如，目标虚拟角色进行移动可以导致角色受到攻击而导致生命值下降，从而得到目标虚拟角色在当前时刻下的生命值。
然后，游戏客户端根据目标虚拟角色在前一时刻下的角色属性值以及在当前时刻下的角色属性值，确定属性值变化幅度。前一时刻指的是游戏客户端对目标虚拟角色进行控制之前的时刻。根据两个时刻下角色属性值的比较，可以确定属性值变化幅度。
进一步地，根据属性值变化幅度，确定场景识别结果。例如，若生命值发生大幅度降低，可以确定场景识别结果为受到大量伤害，以便后续触发终端上连接的物理设备振动来提示用户。
在本申请的实施例中，通过目标虚拟角色的属性值的变化来确定场景识别结果，从而能够在云游戏过程中及时地确定场景识别结果，提升场景识别的粒度，从而更精确地确定需要进行振动反馈情况。
在本申请一个实施例中，基于上述实施例，振动反馈信息包括振动反馈类型，所述云应用客户端控制所述物理设备，执行所述振动反馈类型所指示的振动反馈操作。
具体地，不同物理设备所支持的振动反馈类型可能不同。例如，手机和其他游戏外设可以支持振动特效的强度和种类会有所不同。游戏客户端在生成振动反馈信息时，根据记录的物理设备所支持的振动反馈类型以及识别出的场景需要进行的振动反馈，来生成振动反馈信息。
云游戏客户端在接收到振动反馈信息后，在需要进行振动反馈时，则控制物理设备执 行振动反馈类型所指示的振动反馈操作。例如，游戏客户端判断当前场景需要进行一次长振动和多次短振动，，则生成包括长振动特效类型和短振动特效类型的振动反馈信息，而云游戏客户端在接收到该振动反馈信息后，则控制手机进行长振动和短振动。
在本申请的实施例中，通过在振动反馈信息中限定振动反馈类型，从而可以进行不同种类的振动反馈，提升了振动反馈的多样性。
在本申请一个实施例中，基于上述实施例，振动反馈信息包括振动反馈的时间参数，时间参数用于指示进行振动反馈的时间段，所述云应用客户端控制所述物理设备，在所述时间参数所指示的时间段内，执行振动反馈操作。
具体地，振动反馈的时间参数可以包括振动的开始时刻和持续时间，振动的开始时刻和持续时间可以按照不同场景的需求而定。其中，振动的开始时刻通常是一个相对时刻，其相对于对应的视频流的开始时刻设置，从而确定在何时开始执行振动反馈操作。
例如，振动特效的开始时刻定为10ms，则云游戏客户端在播放对应的视频流10ms之后，开始控制物理设备进行振动。振动的持续时间决定了振动反馈的结束时刻。在一个实施例中，可以不包括振动持续时间，而采用预先预定的统一的持续时间来代替。
应理解的是，振动的持续时间不代表在持续时间内振动反馈动作需要持续进行，例如振动反馈动作可以是间歇振动，则在持续时间内，物理设备会间歇振动，而总时长达到持续时间所限定的数值。
在本申请的实施例中，振动反馈信息包括振动反馈的时间参数，从而可以在播放视频流时，调整振动反馈操作的开始时刻和结束时刻，从而更加灵活地触发振动反馈操作，提升方案灵活性。
在本申请一个实施例中，基于上述实施例，振动反馈信息包括振动反馈的执行频率，所述云应用客户端控制所述物理设备，按照所述振动反馈的执行频率，执行所述振动反馈操作。
具体地，振动反馈的执行频率规定了在振动反馈操作结束时，振动反馈涉及的动作所需要达到的频率。振动反馈的执行频率可以按照不同场景的需求而定，例如紧迫程度的不同可以设定不同的振动执行频率。
例如，振动执行频率表示在振动反馈结束时需要达到的振动频率，可以将取值范围设为0到100的相对值。在一个实施例中，在上一个时刻执行振动反馈可能与当前时刻要执行的振动反馈出现连续，如果执行频率不同，则可以对频率进行调整，使其连续变化，而不需要停止后再重新进行。例如，上一个时刻的振动频率为50，而当前时刻需要执行的振动频率为70，则可以将振动频率按照预设步长从50均匀调整到70。
对于第一个时段，或者对于上一个时刻未进行同类型的振动反馈的情况，则默认上一个时刻的执行频率为0。
在本申请的实施例中，振动反馈信息包括振动反馈的执行频率，从而可以根据不同的场景展现进行不同频率的振动反馈，从而营造出不同的氛围，提升特效的灵活性和表现力。
在本申请一个实施例中，基于上述实施例，振动反馈信息包括振动反馈的执行强度，所述云应用客户端控制所述物理设备，按照所述振动反馈的执行强度，执行所述振动反馈操作。
具体地，振动反馈的执行强度规定了在振动反馈操作结束时，振动反馈涉及的动作所需要达到的振动强度。振动反馈的执行强度可以按照不同场景的需求而定，例如，依照严重程度的不同，可以设定不同的振动执行强度。
执行强度可以表示为振幅等信息，可以将取值范围设为0到100的相对值。
在一个实施例中，在上一个时刻执行的振动反馈可能与当前时刻要执行的振动反馈出现连续，如果执行强度不同，则可以对强度进行调整，使其连续变化，而不需要停止后再重新进行。
例如，上一个时刻的振动强度为50，而当前时刻需要执行的振动强度为70，则可以将振动强度按照预设步长从50均匀调整到70。
对于第一个时段，或者对于上一个时刻未进行同类型的振动反馈的情况，则默认上一个时刻的执行强度为0。
在本申请的实施例中，振动反馈信息包括振动反馈的执行强度，从而可以根据不同的场景展现进行不同强度的振动反馈，从而与游戏的内容更加符合，提升振动反馈的多样性。
在本申请一个实施例中，基于上述实施例，上述步骤S304，向所述云应用客户端发送所述云应用场景对应的视频流以及所述振动反馈信息，包括：
向云应用服务器发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用服务器根据振动协议对所述振动反馈信息进行打包，并将打包结果和所述视频流发送给所述云应用客户端。
在本实施例中，应用客户端可以运行在单独的服务器上，而云应用服务器则是另一台服务器。应用客户端的服务器与云应用服务器通过有线或者无线网络进行通信。应用客户端将渲染好的云应用场景对应的视频流以及所生成的振动反馈信息发送给云应用服务器，而云应用服务器则根据振动协议对振动反馈信息进行打包，并将打包结果和视频流发送给云应用客户端。
具体地，对于振动特效的振动协议，其定义可以如下表1所示：
Figure PCTCN2022106151-appb-000001
表1，振动特效协议
云应用服务器根据应用客户端发送的振动反馈信息，按照上述的协议将振动特效打包为开始时刻、持续时间、振动频率和振动强度组成的数据包，然后和视频流一起发送给云应用客户端，从而进行振动特效的控制。
在本申请的实施例中，应用客户端通过云应用服务器将视频流以及振动反馈信息发送给云应用客户端，使得多个应用客户端可以共享同一个云应用服务器，从而降低了云应用运营成本。
下面将对本申请中基于云应用的设备控制方法进行介绍，该方法应用于云应用客户端，图7所示为本申请另一实施例提供的基于云应用的设备控制方法的流程图，包括以下 步骤：
步骤S701，建立与云应用服务器之间的通信连接；
步骤S702，接收云应用服务器发送的云应用场景对应的视频流和振动反馈信息，振动反馈信息是由服务器上的应用客户端根据云应用场景生成的；
步骤S703，播放云应用场景对应的视频流，并根据振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作。
具体地，云应用客户端建立与云应用服务器之间的通信连接。该通信建立过程通常是由应用客户端发起的。云应用客户端在登录云应用平台后启动云应用，云应用平台则会在服务器上启动对应的应用客户端，应用客户端则主动与云应用客户端建立通信。云应用客户端则接收应用客户端发送的通信建立请求，并且在本地保存，以用于后续发送控制信息和各类响应消息。
在建立通信后，云应用客户端则会从云应用服务器接收云应用场景对应的视频流和振动反馈信息。以云游戏为例，云游戏中游戏的进行是由云游戏客户端来控制，服务器上的游戏客户端则根据用户的操作来执行游戏内容，并且将云游戏的游戏场景渲染成视频流，还根据游戏场景的内容生成振动反馈信息。云游戏客户端会向用户播放云游戏场景的视频流，以便向用户展示游戏画面和操作结果来进行后续的游戏，并且会控制与云游戏客户端相关联的物理设备执行振动反馈操作。例如，根据振动反馈信息中的振动信息来控制手机进行振动。
在本申请的实施例中，云应用客户端与云应用服务器建立通信连接，并且根据接收到的视频流和振动反馈信息来展示应用内容并且执行振动反馈操作，从而能够实现在云应用的场景下，由服务器上的应用客户端直接触发云应用客户端关联的物理设备的振动，避免了根据音频或者视频内容进行振动时产生的与应用内容不匹配的情况，提升了振动反馈的准确性。
在本申请一个实施例中，基于上述实施例，上述步骤，建立与云应用服务器之间的通信连接，可以包括如下步骤：
接收所述云应用服务器发送的通信建立请求；
根据所述通信建立请求，查询所述物理设备所支持的振动反馈类型和振动反馈的协议版本；
向所述云应用服务器发送通信响应消息，所述通信响应消息中包含所述振动反馈类型和所述振动反馈的协议版本，以使所述应用客户端根据所述振动反馈类型和所述振动反馈的协议版本，生成所述振动反馈信息。
具体地，以云游戏为例，云游戏客户端接收云游戏服务器发送的通信建立请求。通信建立请求中通常会包含通信所需要的通信参数，云游戏客户端则根据通信建立请求与云游戏服务器建立通信连接。通信建立请求中还会包含特效查询指令，云游戏客户端会根据该查询指令，查询与云游戏客户端相关联的物理设备所支持的振动反馈类型和振动反馈的协议版本。振动反馈类型主要用于标识物理设备能够进行的振动种类，例如振动长短、模式等等。协议版本则主要用于规定在定义振动反馈操作时需要用到的参数和通信格式等。
云游戏客户端向云游戏服务器发送通信响应消息，通信响应消息中包含振动反馈类型和振动反馈的协议版本，通信响应消息用于使云游戏服务器建立与云游戏客户端的通信连接。具体地，通信响应消息中可以携带云游戏客户端的通信参数，例如互联网地址等，以供云游戏服务器建立相应的通信连接。
在本申请的实施例中，具体介绍了云游戏客户端与云游戏服务器之间建立通信的具体方式，提升了方案的可实施性。
在本申请一个实施例中，基于上述实施例，与云应用客户端相关联的物理设备还包括第三方设备；所述根据所述振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作，包括：
根据振动反馈信息，生成针对于第三方设备的振动控制指令；
向第三方设备发送振动控制指令，以使第三方设备根据振动反馈信息，执行振动反馈操作。
第三方设备指的是用于进行应用控制和云应用内容展示之外的设备。该第三方设备可以是物料网设备并且与云应用客户端所处的终端设备通过网络通信。云应用客户端根据振动反馈信息，生成针对于第三方设备的振动控制指令。振动控制指令中包括根据振动反馈信息所生成的用于执行振动的参数。例如，振动的开始时刻和结束时刻、展示方式和强度等数值。云应用客户端向第三方设备发送振动控制指令，以使第三方设备根据振动反馈信息，执行振动反馈操作。
以云游戏为例进行说明，假定用户利用手机玩探险类游戏，并且控制角色进入了较颠簸的环境，则游戏客户端发送给云游戏客户端的振动反馈信息中可以包括关于座椅晃动控制的信息，而云游戏客户端根据该振动反馈信息，生成用于座椅晃动控制的振动控制指令，并且发送给屋内的智能座椅，而智能座椅则根据收到的振动控制指令调低进行晃动，从而给用户更加身临其境的感觉。
在本申请的实施例中，云游戏客户端还控制第三方设备来协同进行振动反馈，从而使得振动的展示过程更加丰富，提升振动的展示效果。
应当注意，尽管在附图中以特定顺序描述了本申请中方法的各个步骤，但是，这并非要求或者暗示必须按照该特定顺序来执行这些步骤，或是必须执行全部所示的步骤才能实现期望的结果。附加的或备选的，可以省略某些步骤，将多个步骤合并为一个步骤执行，以及/或者将一个步骤分解为多个步骤执行等。
以下介绍本申请的装置实施，可以用于执行本申请上述实施例中的基于云应用的设备控制方法。图8A示意性地示出了本申请实施例中基于云应用的设备控制装置的组成框图，用于执行上述实施例中服务器上的应用客户端相关的处理。如图8A所示，设备控制装置800包括：
通信建立模块801，用于建立与终端设备上的云应用客户端之间的通信连接，所述云应用客户端用于呈现所接收到的云应用视频流；
场景识别模块802，用于对所述云应用客户端所对应的云应用进行场景识别，得到场景识别结果，所述场景识别结果包括与识别到的云应用场景相关联的媒体内容；
振动反馈信息生成模块803，用于根据所述场景识别结果，生成与所述云应用场景所对应的振动反馈信息；
振动反馈信息发送模块804，用于向所述云应用客户端发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用客户端在呈现所述云应用场景的视频流时，根据所述振动反馈信息，控制与所述云应用客户端相关联的物理设备执行振动反馈操作。
在本申请的一些实施例中，基于以上技术方案，通信建立模块801包括：
通信建立请求发送单元，用于向所述云应用客户端发送通信建立请求；
通信响应消息接收单元，用于接收所述云应用客户端返回的通信响应消息；根据所述 通信响应消息，与所述云应用客户端建立通信连接；
其中，所述通信响应消息中包含所述物理设备所支持的振动反馈类型和振动反馈的协议版本；
场景识别模块802用于，根据所述场景识别结果、以及所述振动反馈类型和所述振动反馈的协议版本，生成所述振动反馈信息。
在本申请的一些实施例中，基于以上技术方案，场景识别模块802包括：
角色属性值获取单元，用于获取所述云应用中的目标虚拟角色在当前时刻下的角色属性值；
场景识别单元，用于根据所述角色属性值，识别所述云应用场景，得到所述场景识别结果。
在本申请的一些实施例中，基于以上技术方案，场景识别模块802包括：
场景标识获取单元，用于根据预设的时间段与场景标识之间的映射关系，获取目标虚拟角色在当前时刻下的场景标识，所述场景标识用于表示所述目标虚拟角色所处的周边环境；
场景识别结果确定单元，用于根据预设的场景标识与场景类型之间的映射关系、以及所述在当前时刻下的场景标识，确定所述场景识别结果。
在本申请的一些实施例中，基于以上技术方案，角色属性值获取单元包括：
角色控制指令接收子单元，用于接收所述云应用客户端发送的针对所述目标虚拟角色的角色控制指令；
控制虚拟角色子单元，用于根据所述角色控制指令，对所述目标虚拟角色进行控制，得到所述目标虚拟角色在所述当前时刻下的角色属性值；
场景识别单元包括：
属性值变化幅度确定子单元，用于根据所述目标虚拟角色在前一时刻下的角色属性值以及在所述当前时刻下的角色属性值，确定属性值变化幅度；
识别结果确定子单元，用于根据所述属性值变化幅度，确定所述场景识别结果。
在本申请的一些实施例中，基于以上技术方案，所述振动反馈信息包括振动反馈类型，所述云应用客户端控制所述物理设备，执行所述振动反馈类型所指示的振动反馈操作。
在本申请的一些实施例中，基于以上技术方案，所述振动反馈信息包括振动反馈的时间参数，所述时间参数用于指示进行振动反馈的时间段，所述云应用客户端控制所述物理设备，在所述时间参数所指示的时间段内，执行振动反馈操作。
在本申请的一些实施例中，基于以上技术方案，所述振动反馈信息中包括振动反馈的执行频率，所述云应用客户端控制所述物理设备，按照所述振动反馈的执行频率，执行所述振动反馈操作。
在本申请的一些实施例中，基于以上技术方案，所述振动反馈信息中包括振动反馈的执行强度，所述云应用客户端控制所述物理设备，按照所述振动反馈的执行强度，执行所述振动反馈操作。
在本申请的一些实施例中，基于以上技术方案，振动反馈信息发送模块804包括：
服务器通信单元，用于向云应用服务器发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用服务器根据振动协议对所述振动反馈信息进行打包，并将打包结果和所述视频流发送给所述云应用客户端。
根据本申请实施例的另一个方面，提供一种基于云应用的设备控制装置。图8B示意 性地示出了本申请另一实施例中基于云应用的设备控制装置的组成框图，用于执行上述实施例中云应用客户端相关的处理。如图8B所示，设备控制装置810包括：
通信连接建立模块811，用于建立与云应用服务器之间的通信连接；
振动反馈信息接收模块812，用于接收所述云应用服务器发送的云应用场景对应的视频流和振动反馈信息，所述振动反馈信息是由服务器上的应用客户端根据所述云应用场景生成的；
播放模块813，用于播放所述云应用场景对应的视频流；
振动反馈执行模块814，用于根据所述振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作。
在本申请的一些实施例中，基于以上技术方案，通信连接建立模块811包括：
通信建立请求接收单元，用于接收所述云应用服务器发送的通信建立请求；
振动反馈查询单元，用于根据所述通信建立请求，查询所述物理设备所支持的振动反馈类型和振动反馈的协议版本；
通信响应消息发送单元，用于向所述云应用服务器发送通信响应消息，所述通信响应消息中包含所述振动反馈类型和所述振动反馈的协议版本，以使所述应用客户端根据所述振动反馈类型和所述振动反馈的协议版本，生成所述振动反馈信息。
在本申请的一些实施例中，基于以上技术方案，与所述云应用客户端相关联的物理设备还包括第三方设备；所述振动反馈执行模块还包括：
振动控制指令生成模块，根据所述振动反馈信息，生成针对于所述第三方设备的振动控制指令；
振动控制指令发送模块，用于向所述第三方设备发送所述振动控制指令，以使所述第三方设备根据所述振动反馈信息执行振动反馈操作。
需要说明的是，上述实施例所提供的装置与上述实施例所提供的方法属于同一构思，其中各个模块执行操作的具体方式已经在方法实施例中进行了详细描述，此处不再赘述。
图9示出了适于用来实现本申请实施例的电子设备的结构示意图。
需要说明的是，图9示出的电子设备900仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图9所示，电子设备900包括中央处理单元(Central Processing Unit，CPU)901，其可以根据存储在只读存储器(Read-Only Memory，ROM)902中的程序或者从储存部分908加载到随机访问存储器(Random Access Memory，RAM)903中的程序而执行各种适当的动作和处理。在RAM 903中，还存储有系统操作所需的各种程序和数据。CPU 901、ROM 902以及RAM 903通过总线904彼此相连。输入/输出(Input/Output，I/O)接口905也连接至总线904。
以下部件连接至I/O接口905：包括键盘、鼠标等的输入部分906；包括诸如阴极射线管(Cathode Ray Tube，CRT)、液晶显示器(Liquid Crystal Display，LCD)等以及扬声器等的输出部分907；包括硬盘等的储存部分908；以及包括诸如LAN(Local Area Network，局域网)卡、调制解调器等的网络接口卡的通信部分909。通信部分909经由诸如因特网的网络执行通信处理。驱动器910也根据需要连接至I/O接口905。可拆卸介质911，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器910上，以便于从其上读出的计算机程序根据需要被安装入储存部分908。
特别地，根据本申请的实施例，各个方法流程图中所描述的过程可以被实现为计算 机软件程序。例如，本申请的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分909从网络上被下载和安装，和/或从可拆卸介质911被安装。在该计算机程序被中央处理单元(CPU)901执行时，执行本申请的系统中限定的各种功能。
需要说明的是，本申请实施例所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、闪存、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、有线等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本申请的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
通过以上的实施方式的描述，本领域的技术人员易于理解，这里描述的示例实施方式可以通过软件实现，也可以通过软件结合必要的硬件的方式来实现。因此，根据本申请实施方式的技术方案可以以软件产品的形式体现出来，该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM，U盘，移动硬盘等)中或网络上，包括若干指令以使得一台计算设备(可以是个人计算机、服务器、触控终端、或者网络设备等)执行根据本申请实施方式的方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本申请的其它 实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (20)

  1. 一种基于云应用的设备控制方法，由电子设备执行，包括：
    建立与终端设备上的云应用客户端之间的通信连接，所述云应用客户端用于呈现所接收到的云应用视频流；
    对所述云应用客户端所对应的云应用进行场景识别，得到场景识别结果，所述场景识别结果包括与识别到的云应用场景相关联的媒体内容；
    根据所述场景识别结果，生成与所述云应用场景所对应的振动反馈信息；及，
    向所述云应用客户端发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用客户端在呈现所述云应用场景的视频流时，根据所述振动反馈信息，控制与所述云应用客户端相关联的物理设备执行振动反馈操作。
  2. 根据权利要求1所述的方法，其中，所述建立与终端设备上的云应用客户端之间的通信连接，包括：
    向所述云应用客户端发送通信建立请求；
    接收所述云应用客户端返回的通信响应消息；
    根据所述通信响应消息，与所述云应用客户端建立通信连接；
    其中，所述通信响应消息中包含所述物理设备所支持的振动反馈类型和振动反馈的协议版本；
    所述根据所述场景识别结果，生成与所述云应用场景所对应的振动反馈信息，包括：
    根据所述场景识别结果、以及所述振动反馈类型和所述振动反馈的协议版本，生成所述振动反馈信息。
  3. 根据权利要求1所述的方法，其中，所述对所述云应用客户端所对应的云应用进行场景识别，得到场景识别结果，包括：
    获取所述云应用中的目标虚拟角色在当前时刻下的角色属性值；
    根据所述角色属性值，识别所述云应用场景，得到所述场景识别结果。
  4. 根据权利要求1所述的方法，其中，所述对所述云应用客户端所对应的云应用进行场景识别，得到场景识别结果，包括：
    根据预设的时间段与场景标识之间的映射关系，获取目标虚拟角色在当前时刻下的场景标识，所述场景标识用于表示所述目标虚拟角色所处的周边环境；
    根据预设的场景标识与场景类型之间的映射关系、以及所述在当前时刻下的场景标识，确定所述场景识别结果。
  5. 根据权利要求3所述的方法，其中，所述获取所述云应用中的目标虚拟角色在当前时刻下的角色属性值，包括：
    接收所述云应用客户端发送的针对所述目标虚拟角色的角色控制指令；
    根据所述角色控制指令，对所述目标虚拟角色进行控制，得到所述目标虚拟角色在所述当前时刻下的角色属性值；
    所述根据所述角色属性值，识别所述云应用场景，得到所述场景识别结果，包括：
    根据所述目标虚拟角色在前一时刻下的角色属性值以及在所述当前时刻下的角色属性值，确定属性值变化幅度；
    根据所述属性值变化幅度，确定所述场景识别结果。
  6. 根据权利要求1所述的方法，其中，所述振动反馈信息包括振动反馈类型，所述云应用客户端控制所述物理设备，执行所述振动反馈类型所指示的振动反馈操作。
  7. 根据权利要求1所述的方法，其中，所述振动反馈信息包括振动反馈的时间参数，所述时间参数用于指示进行振动反馈的时间段，所述云应用客户端控制所述物理设备，在所述时间参数所指示的时间段内，执行振动反馈操作。
  8. 根据权利要求1所述的方法，其中，所述振动反馈信息中包括振动反馈的执行频率，所述云应用客户端控制所述物理设备，按照所述振动反馈的执行频率，执行所述振动反馈操作。
  9. 根据权利要求1所述的方法，其中，所述振动反馈信息中包括振动反馈的执行强度，所述云应用客户端控制所述物理设备，按照所述振动反馈的执行强度，执行所述振动反馈操作。
  10. 根据权利要求1至9中任一项所述的方法，其中，所述向所述云应用客户端发送所述云应用场景对应的视频流以及所述振动反馈信息，包括：
    向云应用服务器发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用服务器根据振动协议对所述振动反馈信息进行打包，并将打包结果和所述视频流发送给所述云应用客户端。
  11. 一种基于云应用的设备控制方法，由电子设备执行，所述方法包括：
    建立与云应用服务器之间的通信连接；
    接收所述云应用服务器发送的云应用场景对应的视频流和振动反馈信息，所述振动反馈信息是由服务器上的应用客户端根据所述云应用场景生成的；及，
    播放所述云应用场景对应的视频流，并根据所述振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作。
  12. 根据权利要求11所述的方法，其中，所述建立与云应用服务器之间的通信连接，包括：
    接收所述云应用服务器发送的通信建立请求；
    根据所述通信建立请求，查询所述物理设备所支持的振动反馈类型和振动反馈的协议版本；
    向所述云应用服务器发送通信响应消息，所述通信响应消息中包含所述振动反馈类型和所述振动反馈的协议版本，以使所述应用客户端根据所述振动反馈类型和所述振动反馈的协议版本，生成所述振动反馈信息。
  13. 根据权利要求12所述的方法，其中，所述物理设备包括第三方设备，所述根据所述振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作，包括：
    根据所述振动反馈信息，生成针对于所述第三方设备的振动控制指令；
    向所述第三方设备发送所述振动控制指令，以使所述第三方设备根据所述振动反馈信息，执行振动反馈操作。
  14. 一种基于云应用的设备控制装置，包括：
    通信建立模块，用于建立与终端设备上的云应用客户端之间的通信连接，所述云应用客户端用于呈现所接收到的云应用视频流；
    场景识别模块，用于对所述云应用客户端所对应的云应用进行场景识别，得到场景识别结果，所述场景识别结果包括与识别到的云应用场景相关联的媒体内容；
    振动反馈信息生成模块，用于根据所述场景识别结果，生成与所述云应用场景所对应的振动反馈信息；及，
    振动反馈信息发送模块，用于向所述云应用客户端发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用客户端在呈现所述云应用场景的视频流时，根据所述振动反馈信息，控制与所述云应用客户端相关联的物理设备执行振动反馈操作。
  15. 根据权利要求14所述的装置，其中，所述通信建立模块包括：
    通信建立请求发送单元，用于向所述云应用客户端发送通信建立请求；
    通信响应消息接收单元，用于接收所述云应用客户端返回的通信响应消息；根据所述通信响应消息，与所述云应用客户端建立通信连接；
    其中，所述通信响应消息中包含所述物理设备所支持的振动反馈类型和振动反馈的协议版本；
    所述场景识别模块用于，根据所述场景识别结果、以及所述振动反馈类型和所述振动反馈的协议版本，生成所述振动反馈信息。
  16. 根据权利要求14或15所述的装置，其中，所述振动反馈信息发送模块用于，向云应用服务器发送所述云应用场景对应的视频流以及所述振动反馈信息，以使所述云应用服务器根据振动协议对所述振动反馈信息进行打包，并将打包结果和所述视频流发送给所述云应用客户端。
  17. 一种基于云应用的设备控制装置，包括：
    通信连接建立模块，用于建立与云应用服务器之间的通信连接；
    振动反馈信息接收模块，用于接收所述云应用服务器发送的云应用场景对应的视频流和振动反馈信息，所述振动反馈信息是由服务器上的应用客户端根据所述云应用场景生成的；
    播放模块，用于播放所述云应用场景对应的视频流；及，
    振动反馈执行模块，用于根据所述振动反馈信息，控制与云应用客户端相关联的物理设备执行振动反馈操作。
  18. 根据权利要求17所述的装置，其中，所述物理设备包括第三方设备，所述振动反 馈执行模块用于，根据所述振动反馈信息，生成针对于所述第三方设备的振动控制指令；向所述第三方设备发送所述振动控制指令，以使所述第三方设备根据所述振动反馈信息，执行振动反馈操作。
  19. 一种电子设备，包括：
    处理器；
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行如权利要求1至13中任一项所述的基于云应用的设备控制方法。
  20. 一种计算机可读介质，其上存储有计算机程序，所述计算机程序被处理器执行时，实现如权利要求1至13中任一项所述的基于云应用的设备控制方法。
PCT/CN2022/106151 2021-07-21 2022-07-18 基于云应用的设备控制方法、装置、电子设备和可读介质 Ceased WO2023001086A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2023561072A JP7670852B2 (ja) 2021-07-21 2022-07-18 クラウドゲームに基づく機器制御方法、装置、電子機器、及びコンピュータプログラム
KR1020237033698A KR102939499B1 (ko) 2021-07-21 2022-07-18 클라우드 애플리케이션에 기초한 디바이스 제어 방법 및 장치, 그리고 전자 디바이스 및 저장 매체
US18/239,665 US12569756B2 (en) 2021-07-21 2023-08-29 Cloud application-based device control method and apparatus, electronic device and readable medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110827404.8 2021-07-21
CN202110827404.8A CN115671711B (zh) 2021-07-21 2021-07-21 基于云游戏的设备控制方法、装置、电子设备和可读介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/239,665 Continuation US12569756B2 (en) 2021-07-21 2023-08-29 Cloud application-based device control method and apparatus, electronic device and readable medium

Publications (1)

Publication Number Publication Date
WO2023001086A1 true WO2023001086A1 (zh) 2023-01-26

Family

ID=84978922

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/106151 Ceased WO2023001086A1 (zh) 2021-07-21 2022-07-18 基于云应用的设备控制方法、装置、电子设备和可读介质

Country Status (5)

Country Link
US (1) US12569756B2 (zh)
JP (1) JP7670852B2 (zh)
KR (1) KR102939499B1 (zh)
CN (2) CN115671711B (zh)
WO (1) WO2023001086A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116036578A (zh) * 2023-01-28 2023-05-02 腾讯科技（深圳）有限公司 数据处理方法、装置及相关设备
WO2024219206A1 (ja) * 2023-04-17 2024-10-24 ソニーグループ株式会社 情報処理装置、情報処理方法、およびプログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160082356A1 (en) * 2013-04-19 2016-03-24 Ukida Kensetsu Co., Ltd. Game system control method and game system
CN111729293A (zh) * 2020-08-28 2020-10-02 腾讯科技（深圳）有限公司 一种数据处理方法、装置及存储介质
CN112354176A (zh) * 2020-11-27 2021-02-12 Oppo广东移动通信有限公司 云游戏实现方法、云游戏实现装置、存储介质与电子设备
CN112755530A (zh) * 2020-12-31 2021-05-07 西安万像电子科技有限公司 一种云游戏的数据处理方法、装置、系统及存储介质

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4473685B2 (ja) * 2004-09-01 2010-06-02 任天堂株式会社 ゲーム装置およびゲームプログラム
US20150052253A1 (en) * 2014-09-22 2015-02-19 Weaved, Inc. Multi-server fractional subdomain dns protocol
US20140004941A1 (en) * 2012-06-29 2014-01-02 Sony Computer Entertainment Inc. Conversion of haptic events into screen events
WO2015162942A1 (ja) * 2014-04-25 2015-10-29 株式会社ソニー・コンピュータエンタテインメント 情報処理装置、情報処理方法、プログラム及び情報記憶媒体
US9999835B2 (en) 2015-02-05 2018-06-19 Sony Interactive Entertainment Inc. Motion sickness monitoring and application of supplemental sound to counteract sickness
US10046236B2 (en) * 2016-06-13 2018-08-14 Sony Interactive Entertainment America, LLC Browser-based cloud gaming
CN106227348A (zh) * 2016-07-27 2016-12-14 惠州Tcl移动通信有限公司 振动系统及其振动方法、穿戴式智能设备
WO2019043781A1 (ja) * 2017-08-29 2019-03-07 株式会社ソニー・インタラクティブエンタテインメント 振動制御装置、振動制御方法、及びプログラム
WO2020008862A1 (ja) * 2018-07-02 2020-01-09 ソニー株式会社 情報処理装置、情報処理方法及び情報処理装置可読記録媒体
US11418560B1 (en) * 2019-06-27 2022-08-16 Cable Television Laboratories, Inc. Media and application aware network architecture
CN111420395B (zh) * 2020-04-08 2023-04-18 网易（杭州）网络有限公司 游戏中的交互方法及装置、可读存储介质、电子设备
CN111957039A (zh) * 2020-09-04 2020-11-20 Oppo（重庆）智能科技有限公司 一种游戏特效实现方法、装置及计算机可读存储介质
CN112090060B (zh) * 2020-09-16 2021-07-13 深圳未来游戏设备科技有限公司 云服务器的游戏外设的通讯方式以及适配方法
CN112044078B (zh) * 2020-10-13 2025-07-15 腾讯科技（深圳）有限公司 虚拟场景应用的接入方法、装置、设备及存储介质
CN117695619A (zh) * 2020-10-29 2024-03-15 瑞典爱立信有限公司 多会话远程游戏渲染
CN112604271B (zh) * 2020-12-07 2024-07-19 杭州雾联科技有限公司 一种云游戏系统控制方法、装置、设备及可读存储介质
CN112657186B (zh) * 2020-12-28 2024-02-02 网易（杭州）网络有限公司 一种游戏的交互方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160082356A1 (en) * 2013-04-19 2016-03-24 Ukida Kensetsu Co., Ltd. Game system control method and game system
CN111729293A (zh) * 2020-08-28 2020-10-02 腾讯科技（深圳）有限公司 一种数据处理方法、装置及存储介质
CN112354176A (zh) * 2020-11-27 2021-02-12 Oppo广东移动通信有限公司 云游戏实现方法、云游戏实现装置、存储介质与电子设备
CN112755530A (zh) * 2020-12-31 2021-05-07 西安万像电子科技有限公司 一种云游戏的数据处理方法、装置、系统及存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116036578A (zh) * 2023-01-28 2023-05-02 腾讯科技（深圳）有限公司 数据处理方法、装置及相关设备
CN116036578B (zh) * 2023-01-28 2023-06-13 腾讯科技（深圳）有限公司 数据处理方法、装置及相关设备
WO2024219206A1 (ja) * 2023-04-17 2024-10-24 ソニーグループ株式会社 情報処理装置、情報処理方法、およびプログラム

Also Published As

Publication number Publication date
US12569756B2 (en) 2026-03-10
JP7670852B2 (ja) 2025-04-30
JP2024514800A (ja) 2024-04-03
CN115671711B (zh) 2023-11-21
KR102939499B1 (ko) 2026-03-13
KR20230150392A (ko) 2023-10-30
CN117861194A (zh) 2024-04-12
CN115671711A (zh) 2023-02-03
US20230398443A1 (en) 2023-12-14

Similar Documents

Publication Publication Date Title
US11213743B2 (en) Method, system and electronic device for achieving remote control of computer game by game controller
US10771565B2 (en) Sending application input commands over a network
KR102798321B1 (ko) 정보 처리 방법, 시스템, 장치, 기기 및 저장 매체
CN109568947A (zh) 一种云游戏控制方法、装置、设备和计算机存储介质
CN112245904B (zh) 支持跨服游戏的方法、系统、存储介质及计算机设备
CN115228077A (zh) 云游戏设备交接
CN102921171B (zh) 智能mid、电视、游戏服务器、多屏游戏系统及方法
US12569756B2 (en) Cloud application-based device control method and apparatus, electronic device and readable medium
CN111643889A (zh) 一种游戏控制方法、装置、存储介质和电子设备
CN112169327A (zh) 一种云游戏的控制方法以及相关装置
CN103647768A (zh) 游戏客户端及其实现方法
CN104144351A (zh) 应用虚拟化平台的视频播放方法和装置
CN121233034A (zh) 一种图像处理方法、装置及计算机可读存储介质
CN113617020B (zh) 游戏控制方法、装置、存储介质、服务器以及终端
CN112206514A (zh) 一种游戏画面渲染方法及装置
CN114653063A (zh) 播放音乐资源的方法、装置、存储介质及电子装置
CN116036578B (zh) 数据处理方法、装置及相关设备
HK40081597B (zh) 基於云游戏的设备控制方法、装置、电子设备和可读介质
WO2023207941A1 (zh) 数据传输方法、装置、电子设备、存储介质及程序产品
CN115145660B (zh) 多媒体播放方法、装置、存储介质及终端设备
HK40081597A (zh) 基於云游戏的设备控制方法、装置、电子设备和可读介质
CN110022497A (zh) 视频播放方法及装置、终端设备及计算机可读存储介质
CN114504806A (zh) 基于iptv云游戏的震动信号传递方法及系统
US20260017431A1 (en) Residual physics system
HK40037780A (zh) 一种云游戏的控制方法以及相关装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22845251

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20237033698

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2023561072

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10/06/2024)

122 Ep: pct application non-entry in european phase

Ref document number: 22845251

Country of ref document: EP

Kind code of ref document: A1